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Abstract 

A  hybrid  numerical  algorithm  combining  the  Gauss  Pseudospectral  Method  (GPM) 
with  a  Generalized  Polynomial  Chaos  (gPC)  method  to  solve  nonlinear  stochastic  op¬ 
timal  control  problems  with  constraint  uncertainties  is  presented.  The  GPM  and  gPC 
have  been  shown  to  be  spectrally  accurate  numerical  methods  for  solving  deterministic 
optimal  control  problems  and  stochastic  differential  equations,  respectively.  The  gPC 
uses  collocation  nodes  to  sample  the  random  space,  which  are  then  inserted  into  the 
differential  equations  and  solved  by  applying  standard  differential  equation  methods. 
The  resulting  set  of  deterministic  solutions  is  used  to  characterize  the  distribution  of 
the  solution  by  constructing  a  polynomial  representation  of  the  output  as  a  function  of 
uncertain  parameters.  Optimal  control  problems  are  especially  challenging  to  solve 
since  they  often  include  path  constraints,  bounded  controls,  boundary  conditions, 
and  require  solutions  that  minimize  a  cost  functional.  Adding  random  parameters 
can  make  these  problems  even  more  challenging.  The  hybrid  algorithm  presented  in 
this  dissertation  is  the  first  time  the  GPM  and  gPC  algorithms  have  been  combined 
to  solve  optimal  control  problems  with  random  parameters.  Using  the  GPM  in  the 
gPC  construct  provides  minimum  cost  deterministic  solutions  used  in  stochastic  com¬ 
putations  that  meet  path,  control,  and  boundary  constraints,  thus  extending  current 
gPC  methods  to  be  applicable  to  stochastic  optimal  control  problems.  The  hybrid 
GPM-gPC  algorithm  was  applied  to  two  concept  demonstration  problems:  a  nonlin¬ 
ear  optimal  control  problem  with  multiplicative  uncertain  elements  and  a  trajectory 
optimization  problem  simulating  an  aircraft  flying  through  a  threat  field  where  exact 
locations  of  the  threats  are  unknown.  The  results  show  that  the  expected  value,  vari¬ 
ance,  and  covariance  statistics  of  the  polynomial  output  function  approximations  of 
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the  state,  control,  cost,  and  terminal  time  variables  agree  with  Monte-Carlo  simula¬ 
tion  results  while  requiring  on  the  order  of  to  the  number  of  collocation 

points  and  computation  time,  ft  was  shown  that  the  hybrid  algorithm  demonstrated 
an  ability  to  effectively  characterize  how  the  solutions  to  optimization  problems  vary 
with  uncertainty,  and  has  the  potential  with  continued  development  and  availability 
of  more  powerful  computer  workstations,  to  be  a  powerful  tool  applicable  to  more 
complex  control  problems  of  interest  to  the  Department  of  Defense. 
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HYBRID  SOLUTION  OF  STOCHASTIC  OPTIMAL  CONTROL  PROBLEMS 
USING  GAUSS  PSEUDOSPECTRAL  METHOD  AND  GENERALIZED 
POLYNOMIAL  CHAOS  ALGORITHMS 

I.  Introduction 


1.1  Introduction 

Finding  solutions  to  optimization  problems  is  essential  to  planning  and  conduct¬ 
ing  military  operations.  Virtually  every  application  of  United  States  Air  Force 
(USAF)  air-  and  space-based  weapons  systems,  ranging  from  Remotely  Piloted  Air¬ 
craft  (RPA),  manned  fighter,  bomber,  and  transport  aircraft,  to  satellite  systems  is 
planned  to  maximize  desired  effects  while  minimizing  costs  and  risks.  Some  real-world 
examples  of  Trajectory  Optimization  (TO)  and  Optimal  Control  (OC)  problems  are: 

•  Find  the  optimal  orbit  and  body  orientation  that  allows  an  RPA  to  maximize 
the  time  its  camera  has  eyes  on  target. 

•  Find  the  flight  path  through  a  threat- rich  environment  that  minimizes  a  bomber’s 
exposure  to  threats  while  conducting  a  bombing  run. 

•  Find  the  orbit  and  thrust  (throttle)  inputs  that  maximize  loiter  time  of  an  air 
refueling  or  Intelligence,  Surveillance,  and  Reconnaissance  (ISR)  platform. 

•  Find  the  optimal  thrust  profile  that  minimizes  fuel  expenditure  during  satellite 
orbital  transfer. 

•  Find  the  best  path  through  and  urban  environment  that  takes  a  vehicle  from 
one  point  to  another  in  minimum  time  while  avoiding  obstacles. 

•  Find  the  route  that  allows  an  attack  aircraft  to  strike  the  maximum  number  of 
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targets  in  minimum  distance  traveled  (military  version  of  the  travelling  salesman 
problem). 

Real-world  TO  and  OC  problems  like  these  are  typically  solved  using  deterministic 
models,  which  are  often  simplified  representations  of  a  system’s  true  physics  that 
neglect  several  potential  sources  of  uncertainty.  Higher-fidelity  models  may  be  cost- 
prohibitive  to  develop,  not  feasible  due  to  unknown  physics,  or  difficult  to  use  in 
engineering  work.  Additionally,  virtually  all  sensors  used  to  measure  system  states, 
for  example  speed,  altitude,  latitude,  longitude,  etc.,  are  corrupted  by  noise  thus 
inducing  measurement  errors  that  can  degrade  the  validity  of  TO  or  OC  solutions. 
The  environment  can  also  introduce  randomness  through  phenomena  such  as  wind 
gusts  and  turbulence.  Therefore,  all  real-world  problems  have  uncertainties  that  can 
make  it  difficult  to  fold  effective  solutions  and  cause  deterministic  methods  to  fail  to 
meet  objectives. 

Nonlinear  optimization  problems  are  typically  too  complex  to  use  analytical  solu¬ 
tion  techniques  necessitating  use  of  numerical  methods.  When  stochastic  elements  are 
included,  numerical  methods  become  essentially  the  only  way  to  solve  these  problems. 
Recently,  stochastic  nonlinear  problems  have  become  an  area  of  research  emphasis  in 
the  Air  Force  Research  Laboratory  (AFRL)  both  in  terms  of  basic  research  and  ap¬ 
plied  research  to  address  Micro  Air  Vehicle  (MAV)  control  challenges.  In  a  briefing 
given  at  the  2008  International  Symposium  on  Unmanned  Aerial  Vehicles,  AFRL 
described  its  MAV  research  program  [2]  highlighting  several  “fundamental  scientific 
challenges,”  including  “unsteady  aerodynamics  at  low  Reynolds  number[s]”  and  listed 
several  engineering  and  design  challenges  including: 

•  Determine  what  aerodynamic  models  should  be  develop  ed/used  for  inner-loop 
flight  control 

•  Gust  tolerance  for  MAV  operations  near  walls  and  other  obstacles 
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•  Stabilization  of  low  inertial  flight  vehicles  [small,  lightweight]  in  high  distur¬ 
bance,  gusty  environments 

•  Nonlinear  responses 

AFRL’s  briefing  indicates  that  MAVs  are  nonlinear  systems  subject  to  random  (stochas¬ 
tic)  elements  such  as  gusts  and  model  uncertainties  that  are  excellent  candidates  for 
future  application  of  the  stochastic  optimal  control  method  described  in  this  disser¬ 
tation.  A  2009  Air  Force  Office  of  Scientific  Research  (AFOSR)  briefing  describes 
its  dynamics  and  control  research  portfolio  where  the  recurring  theme  is  to  investi¬ 
gate  numerical  methods  to  solve  nonlinear  stochastic  optimal  control  problems  [64]. 
Combined,  these  presentations  emphasize  that  numerical  tools  are  necessary  to  solve 
stochastic  nonlinear  optimization  problems  to  support  MAV  control  development  and 
solve  other  real-world  problems. 

Two  pseudospectral  numerical  methods,  the  Gauss  Pseudospectral  Method  (GPM) 
and  Generalized  Polynomial  Chaos  (gPC),  may  both  be  useful  in  addressing  stochas¬ 
tic  optimization  problems  like  those  of  interest  to  AFOSR  and  AFRL.  The  GPM  is 
a  powerful  deterministic  numerical  method  for  solving  both  linear  and  nonlinear  TO 
and  OC  problems.  It  is  a  direct  solution  method  that  uses  pseudospectral  discretiza¬ 
tion  and  Gaussian  quadrature  to  recast  the  continuous-time  problem  into  a  nonlinear 
programming  (NLP)  problem  that  can  be  solved  using  existing  NLP  solvers.  The 
gPC,  sometimes  referred  to  as  a  deterministic  sampling  method ,  is  an  equally  power¬ 
ful  method  for  solving  stochastic  differential  equations  (SDE)  where  collocation  points 
in  the  random  domain  are  used  as  inputs  to  stochastic  differential  equations  result¬ 
ing  in  a  set  of  deterministic  problems  that  can  be  solved  using  existing  differential 
equation  solvers.  The  set  of  sampled  deterministic  solutions  are  then  used  in  the  gPC 
expansion  to  quantify  the  effects  of  the  uncertain  parameters  on  the  solution. 
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1.2  Research  Objective  and  Technical  Contribution 


The  objective  of  this  research  is  to  develop  a  new  method  of  analyzing  the  effects 
of  uncertain  parameters  on  optimal  control  and  trajectory  optimization  problem  so¬ 
lutions  and  apply  it  to  two  representative  optimization  problems  to  demonstrate  its 
ability  to  quantify  the  effects  of  uncertainty  on  the  solution.  The  approach  used  in 
this  research  is  the  first  time  the  GPM  and  the  collocation  form  of  the  gPC  have 
been  combined  to  form  a  hybrid  algorithm.  Using  the  GPM  in  the  hybrid  scheme 
provides  spectrally  accurate  minimum  cost  solutions  to  the  sampled  deterministic 
problems  that  meet  specified  constraints.  This  combination  of  deterministic  and 
stochastic  pseudospectral  methods  provides  a  new  numerical  algorithm  for  address¬ 
ing  nonlinear  optimization  problems  where  model,  measurement,  and  environmental 
disturbance  uncertainties  may  be  included  in  the  formulation,  and  their  effects  as¬ 
sessed  by  constructing  distribution  functions  showing  dependence  of  the  solutions 
on  uncertain  parameters.  Thus,  the  main  technical  contribution  of  this  research  is 
to  extend  current  gPC  methods  to  be  applicable  to  stochastic  optimal  control  and 
trajectory  optimization  problems. 

1.3  Concept  Demonstration  Problem  Descriptions 

The  hybrid  numerical  algorithm  combining  the  GPM  and  gPC  methods  presented 
in  this  document  is  applied  to  two  types  of  optimization  problems  to  demonstrate  its 
potential  to  quantify  the  effects  of  uncertainty  on  TO  and  OC  problem  solutions. 

The  first  is  a  general  OC  problem  taken  from  a  textbook  and  modified  by  adding 
Gaussian  random  parameters  effecting  the  state  variables.  The  problem  has  nonlinear 
dynamics,  quadratic  Lagrange  performance  index,  fixed  final  state  and  time,  and 
no  path  or  control  constraints.  The  objective  of  this  problem  is  to  investigate  the 
feasibility  of  using  the  hybrid  algorithm  to  solve  optimization  problems  by  applying 
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it  to  a  very  challenging  nonlinear  problem  where  uncertainties  have  significant  effects 
on  the  solution. 

The  second  problem,  and  really  the  main  focus  of  this  research,  is  based  on  a 
more  operationally  representative  mission  planning  scenario  that  is  of  interest  to 
United  States  Strategic  Command  (USSTRATCOM).  The  problem  was  formulated 
in  response  to  USSTRATCOM’s  interest  in  using  solutions  calculated  by  the  method 
presented  in  this  work  to  initialize  their  mission  planning  software,  with  hopes  of 
generating  mission  plans  more  efficiently.  The  sample  problem  is  a  trajectory  opti¬ 
mization  problem  where  the  objectives  are  to  find  the  path  through  a  two-dimensional 
space  that  minimizes  the  probability  a  vehicle  will  be  killed  by  lethal  threats  whose 
locations  are  uncertain,  and  then  to  quantify  the  effects  those  uncertainties  have  on 
the  solution  by  estimating  the  statistical  properties.  A  generic  two-dimensional  Du- 
bins  model  consisting  of  three  states  and  one  bounded  control  is  used  in  formulating 
the  sample  problem.  The  Dubins  model  was  chosen  in  lieu  of  a  specific  aircraft  model 
to  focus  on  algorithm  development  while  avoiding  additional  complexities  of  develop¬ 
ing  and  implementing  a  dynamics  model  for  a  particular  vehicle.  Bi-variate  Gaussian 
probability  density  functions  are  used  to  model  lethality  distributions  of  the  threats 
in  the  space,  i.e.  threat  rings ,  and  construct  the  nonlinear  cost  functional  to  be  mini¬ 
mized  in  seeking  solutions  to  the  trajectory  optimization  problem.  Random  variables 
(RV)  are  incorporated  into  the  cost  functional  to  represent  uncertainties  in  the  exact 
center  locations  of  the  threat  rings,  making  the  cost  functional  and  associated  state 
and  control  solutions  themselves  RVs  whose  distribution  properties  are  estimated  by 
the  gPC  expansion  coefficients. 

In  both  concept  demonstration  problems,  applying  the  hybrid  algorithm  results  in 
quantification  of  the  effects  of  the  random  parameters  on  the  solution  by  generating  a 
polynomial  distribution  function  of  the  random  inputs  whose  coefficients  are  used  to 
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calculate  expected  value,  variance,  and  covariance  properties.  An  additional  benefit  is 
that  if  uncertain  variables  become  certain,  the  distribution  function  can  be  evaluated 
at  those  points  to  provide  specific  optimal  solutions. 

1.4  Document  Organization 

This  document  is  organized  to  present  work  related  to  solving  optimal  control 
problems,  discuss  application  of  the  algorithm  to  two  concept  demonstration  prob¬ 
lems,  and  to  recommend  future  research  activities.  Chapter  II  presents  work  found  in 
the  literature  related  to  solving  optimal  control  problems  and  stochastic  differential 
equations.  It  begins  with  a  brief  history  of  the  development  of  optimal  control  the¬ 
ory,  defines  an  optimal  control  problem  and  describes  the  analytical  solution  approach 
based  on  calculus  of  variations  theory,  and  discusses  numerical  methods  available  to 
solve  optimal  control  problems,  including  a  detailed  description  of  the  Gauss  Pseu- 
dospectral  Method.  Readers  who  are  familiar  with  optimal  control  theory  may  wish 
to  skip  directly  to  the  Gauss  Pseudospectral  Method  discussion  in  section  2. 3. 3. 2. 
The  chapter  ends  with  a  summary  of  methods  used  to  solve  stochastic  differential 
equations  and  a  detailed  description  of  the  Generalized  Polynomial  Chaos  method. 
Chapter  III  gives  a  brief  presentation  of  the  hybrid  algorithm  combining  the  GPM 
and  gPC  method  as  it  is  applied  to  the  demonstration  problems  in  later  chapters. 
The  algorithm  is  presented  as  a  step-by-step  process  and  a  code  map  of  the  Mat- 
LAB®implementation  is  included.  Formulations  and  results  of  applying  the  hybrid 
algorithm  to  the  concept  demonstration  problems  are  presented  in  Chapters  IV  and 
V,  respectively.  The  document  concludes  with  recommendations  for  further  research 
in  Chapter  VI. 
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II.  Related  Concepts 


The  objective  of  OC  theory  is  “to  determine  the  control  signals  that  will  cause 
a  process  to  satisfy  the  physical  constraints  and  at  the  same  time  minimize 
(or  maximize)  some  performance  criterion”  [54].  The  performance  criterion  is  defined 
by  an  integral  performance  index,  also  referred  to  as  a  cost  functional,  that  is  to  be 
minimized  and  the  physical  constraints  are  the  system  or  process  dynamics  to  be 
controlled.  The  idea  is  to  cause  the  system  to  follow  some  ideal,  or  optimal,  state 
trajectory  by  solving  for  the  input,  or  control,  that  causes  the  system  to  follow  the 
trajectory  that  minimizes  the  performance  index.  The  performance  index  can  be  cho¬ 
sen  to  solve  a  variety  of  problems  such  as  minimum  time,  minimum  fuel  consumption, 
minimum  energy  expended  or  control  effort  used,  minimum  error  between  desired  and 
achieved  end  state  (terminal  control),  minimum  error  between  desired  and  achieved 
path  (trajectory  following),  and  so  on.  This  chapter  focuses  on  the  body  of  work 
related  to  solving  OC  problems.  The  chapter  begins  with  a  brief  historical  summary 
of  the  development  of  OC  theory.  It  continues  by  defining  an  OC  problem  and  the 
classical  analytical  method  of  solving  the  problem  stemming  from  the  historical  devel¬ 
opment.  The  chapter  continues  with  a  summary  of  numerical  techniques  used  to  solve 
OC  problems,  culminating  with  a  description  of  a  powerful  numerical  technique,  the 
Pseudospectral  Method  (PSM)  for  approximating  the  solution.  Next,  related  work 
is  presented  on  solving  SDEs  along  with  current  research  applications.  The  chapter 
concludes  by  linking  the  theory  of  OC  and  the  methods  of  solving  SDEs  that  appear 
applicable  to  solving  stochastic  optimal  control  problems,  which  will  set  the  stage  for 
the  work  to  follow  in  subsequent  chapters. 
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2.1  A  Brief  History  of  Optimal  Control  Theory 


Queen  Dido  of  Carthage  is  generally  thought  of  as  to  have  solved  one  of  the  first 
optimization  problems.  Legend  has  it  that  she  was  promised  the  amount  of  land 
that  could  be  enclosed  using  a  bull’s  hide.  Her  solution  was  to  cut  the  hide  into 
thin  strings  and  tie  them  together  to  form  a  circle  [54],  To  the  ancient  Greeks,  the 
circle  had  to  be  the  best  solution  because  in  Aristotle’s  thinking  it  represented  the 
“perfect  figure”  [85].  She  solved  this  problem  before  the  mathematical  development 
of  the  Calculus  of  Variations  (CV),  which  proves  that  the  optimal  closed  curve  that 
maximizes  the  area  is  in  fact  a  circle,  and  forms  the  mathematical  basis  for  solving 
OC  problems. 

The  first  OC  problem  was  posed  by  Johann  Bernoulli  (1667-1748)  in  1696  when  he 
issued  a  challenge  to  his  contemporaries  to  solve  the  brachystochrone  (from  two  Greek 
words  meaning  “shortest”  and  “time”)  problem  [85].  Bernoulli’s  brachystochrone 
problem  was  really  a  re-statement  of  the  problem  posed  by  Galileo  Galilei  (1564- 
1642)  in  1638  whose  thoughts  on  the  solution  later  proved  to  be  incorrect  [16,85]. 
The  goal  of  the  brachystochrone  problem  was  to  “find  the  shape  of  a  wire  such  that  a 
bead  sliding  along  it  traverses  the  distance  between  the  two  end  points  in  minimum 
time”  [16].  Bernoulli  noted  in  his  challenge  that  the  shortest  distance  between  the 
end  points  would  be  a  straight  line,  but  the  minimum  time  to  travel  from  one  end 
of  the  wire  to  the  other  would  not  be  achieved  by  following  the  straight  line  path. 
The  five  mathematicians  who  solved  the  problem  were  Johann  Bernoulli  himself, 
his  brother  Jakob  (1654-1705),  Wilhelm  Gottfried  Leibniz  (1646-1716),  Marquis  de 
l’Hopital  (1661-1704)  and  Isaac  Newton  (1642-1727).  It’s  noteworthy  to  point  out 
that  Jakob  Bernoulli’s  solution  resembled  future  concepts  of  CV,  Hamilton-Jacobi 
theory,  and  Dynamic  Programming  (DP)  [21,85].  This  was  among  the  earliest  of  OC 
problems  since  it  dealt  with  finding  the  control  input,  in  this  case  the  shape  of  the 


curved  wire,  that  controlled  the  behavior  of  a  dynamical  system  [85]. 

Willems  notes  that  things  were  relatively  quiet  after  the  solution  of  Bernoulli’s 
brachystochrone  problem  until  about  the  1960’s  [85].  The  quiet  period  was  probably 
due  to  the  fact  that  a  rigorous  mathematical  structure,  known  today  as  CV,  needed 
to  evolve  in  order  to  solve  OC  problems,  which  was  developed  in  the  18th  and  19th 
centuries  [32], 

Leibniz  and  Newton  are  said  to  have  co-invented  CV.  Leibniz  was  interested  in 
finding  extrema  of  functions  and  published  a  paper  titled  A  new  method  for  the 
determination  of  maxima  and  minima...  in  1684  [55].  Newton  solved  one  of  the  first 
problems  using  CV  when  he  correctly  posed  and  solved  a  problem  to  find  the  shape  of 
a  projectile  moving  through  air  that  resulted  in  minimum  drag  in  1685,  and  published 
in  1694  [16,32,55].  At  the  time,  there  was  a  need  for  the  theory  of  Calculus  to  be 
extended  to  address  more  general  problems  involving  finding  paths,  curves,  or  surfaces 
that  result  in  stationary  values  (maxima  or  minima)  of  functionals  [32,55].  The  work 
of  Leibniz,  Newton,  and  the  solutions  to  Bernoulli’s  brachystochrone  problem  in  1696 
can  be  thought  of  as  the  beginnings  of  CV. 

Many  early  problems  were  shown  to  be  of  similar  form,  generally  defined  by  the 
functional  /  and  boundary  conditions  given  as  [79]: 

1  =  fa  L(q(t),<Kt),t)dt 

q(a)  =  qa  ,  q(b)  =  qb  (2.1) 

a  <  t  <  b 


Addressing  the  problem  in  (2.1)  meant  finding  a  function  q(t)  that  resulted  in  a 
stationary  /,  indicating  either  a  maximum  or  minimum,  while  satisfying  the  bound¬ 
ary  conditions  [55].  Leonard  Euler  (1707-1783)  was  the  first  to  formally  study  this 
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problem  in  the  late  1720’s  and  1730’s  and  published  a  book  in  1744,  referred  to  as 
the  “birth  year  of  the  theory”  of  CV,  entitled  A  method  for  discovering  curved  lines 
that  enjoy  a  maximum  or  a  minimum  property,  or  the  solution  of  the  isoperimetric 
problem  taken  in  the  widest  sense  [32,55].  An  excellent  detailed  discussion  of  Euler’s 
method  can  be  found  in  [47].  Euler’s  main  contribution  to  the  development  of  CV 
theory  was  derivation  of  what  is  known  as  the  Euler  equation.  Euler  found  that  a 
necessary  condition  for  q(t)  satisfying  (2.1)  to  be  a  minimum  curve  had  to  also  satisfy: 


dL  d  \dL 
dq  dt  dq 


0 


(2.2) 


Joseph  Louis  Lagrange  (1736-1813)  extended  Euler’s  work  by  inventing  his  “method 
of  variations”  and  his  multiplier  rule,  published  in  1760  in  a  paper  entitled  Essay  on  a 
new  method  for  determining  maxima  and  minima  for  formulas  of  indefinite  integrals. 
Using  these  methods,  he  studied  the  first  variation,  denoted  as  SI,  of  the  functional 
in  (2.1).  He  concluded  that  the  extremal  solution  q(t)  was  obtained  when  the  first 
variation  vanished,  analogous  to  solving  for  f'(x)  =  0  in  the  case  of  finding  maxima 
or  minima  of  a  function  using  traditional  calculus.  He  showed  that  when  the  first 
variation  vanished,  Euler’s  equation  was  left,  more  convincingly  showing  that  Euler’s 
equation  was  a  necessary  condition  for  finding  the  function  q(t)  that  leads  to  an  ex¬ 
tremal  value  of  I.  Subsequently,  Euler  gave  the  held  the  name  Calculus  of  Variations 
in  honor  of  Lagrange’s  variational  method  and  equation  (2.2)  became  known  as  the 
Euler- Lagrange  equation.  The  significance  of  Lagrange’s  work  was  to  generalize  Eu¬ 
ler’s  geometric  approach  to  applied  problems  by  focusing  on  “algorithmic  aspects  of 
analysis”  based  on  differential  equations  [32,55]. 

In  calculus,  the  second  derivative  test  provides  a  necessary  condition  for  deter¬ 
mining  if  extremal  points  are  maxima  or  minima.  Likewise,  Euler’s  equation  and 
examining  the  first  variation  of  the  functional  only  provides  the  necessary  condition 


10 


for  a  maximum  or  minimum  curve.  The  second  variation,  <52/,  provides  the  analogous 
test  for  determining  maximum  and  minimum  curves  in  CV.  Adrien-Marie  Legendre 
(1752-1833)  was  the  first  to  look  at  the  second  variation  of  the  functional  and  deter¬ 
mined,  without  complete  proof,  that  Lgg  >  0  along  a  minimizing  curve  and  Lgg  <  0 
(subscripts  denote  partial  derivatives)  along  a  maximizing  one,  as  published  in  1786. 
Karl  Gustav  Jacob  Jacobi  (1804-1851)  later  provided  rigorous  proof  that  Lgg  >  0 
along  with  the  Jacobi  condition  provide  sufficiency  for  showing  q(t)  is  a  minimum 
curve  [16,32,55]. 

Sir  William  Rowan  Hamilton  (1805-1865)  was  the  next  to  add  to  the  theory  of 
CV.  In  papers  he  wrote  in  1834  and  1835  he  “showed  that  under  certain  conditions, 
problems  in  mechanics  involving  many  variables  and  constraints  can  be  reduced  to 
an  examination  of  the  partial  derivatives  of  a  single  function”  [16,32],  Hamilton’s 
function  was: 

'H=<p,q>  -L(q,q,t)  (2.3) 

Where  p  =  Lg{q,q,t )  and  the  <  p,q  >  term  denotes  an  inner  product.  Hamilton 
found  that  looking  at  Up  =  q  and  —Jig  =  p  was  equivalent  to  (2.2)  under  the 
assumptions  that  q  is  treated  not  as  an  independent  variable  but  as  a  function  of  q,  p, 
and  t  and  that  the  p  equation  can  be  solved  for  q  [79].  Sussmann  and  Willems  noted 
that  Hamilton  should  have  written  his  function  in  another  way.  Letting  u(t)  =  q(t), 
defining  the  Hamiltonian,  "H,  as: 

H(q,u,p,t)  =<p,u>  -L(q,u,t)  (2.4) 

and  p(t)  as: 

p(t)  =  Lu(q(t),q(t),t)  (2.5) 
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leads  to: 


Kp  =  q 

-nq  =p  (2.6) 

nu  =o 

This  is  a  more  general  result  and  is  directly  equivalent  to  (2.2).  This  result  is  referred 
to  as  the  “control  Hamiltonian”  and  Sussmann  and  Willems  state,  “the  control  ver¬ 
sion  of  the  Hamiltonian  equations  is  equivalent  to  the  Euler-Lagrange  system  under 
completely  general  conditions...”.  Equation  (2.4)  will  be  the  basis  for  finding  the 
analytical  OC  solution  presented  in  the  next  section. 

Hamilton’s  proof  was  lacking  details  that  Jacobi  added  in  1838.  Jacobi  was  doing 
similar  work  at  about  the  same  time  as  Hamilton  that  showed  that  “the  partial 
derivatives  of  the  performance  index  with  respect  to  each  parameter  of  a  family  of 
extremals  (which  today  we  call  ‘states’)  obeyed  a  certain  differential  equation”  [16]. 
Jacobi  cleaned  up  Hamilton’s  results  and  simplified  them  [16,32].  We  now  know 
this  theory  as  Hamilton-Jacobi  theory,  which  became  the  basis  of  DP  developed  by 
Bellman  in  the  1950 ’s. 

In  the  late  19th  and  the  first  half  of  the  20th  centuries,  mathematicians  like  Karl 
Theodor  Wilhelm  Weierstrass  (1815-1897),  Oskar  Bolza  (1857-1942),  and  Gilbert 
Bliss  (1876-1951)  added  mathematical  rigor  to  the  theory  of  CV.  Weierstrass  discov¬ 
ered  a  condition  which  became  the  predecessor  to  maximum  principles  later  stated 
by  Bellman  and  Pontryagin  and  was  the  first  to  develop  a  complete  sufficiency  theo¬ 
rem  for  a  minimum.  Bolza  and  Bliss  built  upon  Weierstrass’  work  at  the  University 
of  Chicago  giving  CV  its  current  rigorous  mathematical  structure,  culminating  in 
publication  of  the  book  Lectures  on  the  Calculus  of  Variations  in  1946  [16,32], 
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The  evolved  CV  theory  was  the  enabling  structure  for  OC  theory  to  progress 
beginning  in  the  1950’s.  OC  theory  is  a  generalization  of  CV  and  allows  nonlinear  and 
non-smooth  functionals  to  be  optimized  [32],  Bryson  gives  an  outstanding  summary 
of  the  roots  of  OC  theory  from  1950  to  1985  in  [16].  His  paper  discusses  the  roots 
of  OC  theory  in  contexts  of  classical  control,  random  processes,  linear  and  nonlinear 
programming,  and  DP.  The  rest  of  the  historical  discussion  of  OC  theory  in  this 
section  is  a  summary  of  Bryson’s  main  points. 

Until  the  years  surrounding  World  War  11  (WWII),  classical  linear  control  meth¬ 
ods,  such  as  Proportional  (P),  Proportional  +  Integral  (PI),  Proportional  +  Integral 
-f  Derivative  (PID),  Lead,  Lag,  and  Lead-Lag  involved  a  degree  of  ad-hoc  design  of 
control  gains.  Control  gains  were  selected  and  adjusted  based  on  observed  perfor¬ 
mance  until  some  acceptable  result  was  achieved,  leaving  a  considerable  degree  of 
subjectiveness  to  control  design.  During  the  WWII  period,  several  methods  were 
developed  to  systematically  choose  control  gains  including  Evans’  Root  Locus  (RL) 
method,  the  Bode  plot  method,  and  the  Nyquist  plot  method.  These  methods,  and 
others,  allowed  designers  to  select  gains  based  on  some  desired  stability  and  perfor¬ 
mance  criteria.  In  the  1960’s,  Kalman  introduced  the  Linear  Quadratic  Regulator 
(LQR)  method  using  an  integral  performance  index  that  placed  quadratic  penalties 
on  output  errors  and  amount  of  control  used  [51,52],  He  used  CV  to  show  that  the 
OC  inputs  can  be  found  using  linear  feedback  of  the  state  variables  and  was  able  to 
apply  this  method  to  time-varying  and  Multi-Input  Multi-Output  (MIMO)  systems. 
Kalman  further  showed  that  the  optimal  state  feedback  gain  matrix  could  be  found 
by  solving  a  backward  Riccati  equation  [16].  His  contribution  was  to  replace  ad-hoc 
gain  selection  with  a  method  to  solve  for  optimal  feedback  control  gains. 

Some  would  say  that  OC  theory  was  truly  born  in  the  1950’s  and  1960’s  with 
the  publication  of  The  Mathematical  Theory  of  Optimal  Processes  [69]  in  1961  by 
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Soviet  mathematician  Lev  Semenovich  Pontryagin  (1908-1988)  and  a  group  of  his 
students  [79].  Pontryagin’s  well  developed  theory  forms  the  basis  for  modern  optimal 
control,  is  applicable  to  linear  and  nonlinear  problems,  and  is  usually  the  benchmark 
for  comparing  results  produced  by  numerical  methods  developed  later.  His  theory 
extended  CV  and  Weierstrass’  necessary  condition  to  address  inequality  constraints 
and  stated  that  a  minimizing  path  must  satisfy  the  Euler-Lagrange  equations  (2.2) 
and  that  the  OC  solution  maximizes  the  Hamiltonian  (2.3)  in  their  bounded  region 
at  each  point  along  the  curve  [16].  Pontryagin  used  a  different  sign  convention  on 
the  Hamiltonian,  thus  the  term  maximum  principle ,  but  the  principle  is  applied  to 
End  minimizing  solutions  of  the  Hamiltonian  and  is  thus  referred  to  as  Pontryagin’s 
Minimum  Principle  (PMP). 

The  field  of  OC  is  closely  tied  to  the  study  of  optimal  signal  filtering.  Models 
representing  a  system’s  dynamics  are  typically  assumed  to  be  noise-free.  That  is  to 
say  that  an  assumption  is  made  that  the  state  of  the  system  represented  by  a  set  of 
differential  equations  is  perfectly  known.  In  practice,  models  are  a  simplification  of 
the  true  physical  nature  of  the  system  and  measurements  of  system  states  are  noise- 
corrupted.  Therefore,  some  method  must  be  implemented  to  extract  the  best  possible 
estimate  of  the  state  with  the  presence  of  model  and  measurement  uncertainties. 
Pioneers  in  the  study  of  optimal  filtering  were  Norbert  Wiener  (1894-1964)  in  the 
1940’s  [84]  and  Ruldolf  Kalman  and  Richard  Bucy  in  the  1950’s  and  1960’s  [51, 
52],  These  researchers  essentially  developed  optimal  techniques  to  minimize  errors 
between  the  actual  measurements  and  the  estimated  system  states  obtained  from  the 
dynamical  models.  Kalman  showed  that  optimal  filter  gains  are  obtained  by  solving 
a  forward  Riccati  equation  and  became  known  as  the  Linear  Quadratic  Estimator 
(LQE)  [16].  The  LQE  combined  with  the  LQR  method  became  known  as  the  Linear 
Quadratic  Gaussian  (LQG)  compensator.  The  LQG  compensator,  an  OC  scheme 
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that  is  applicable  to  linear  time-varying  systems  with  nncorrelated  Gaussian  random 
inputs,  feeds  back  optimal  state  estimates  that  are  then  used  to  solve  the  OC  problem 
which  minimizes  the  quadratic  performance  index  [16]. 

Development  of  LQR,  LQE,  LQG,  and  Wiener  filtering  methods  resulted  in  sig¬ 
nificant  advancements  in  the  OC  held,  ffowever,  these  methods  are  primarily  focused 
on  linear  systems.  The  power  of  the  CV  applied  to  OC  problems  is  that  it  can  be  used 
to  solve  nonlinear  problems.  The  difficulty,  however,  is  that  applying  Euler- Lagrange, 
Hamilton-Jacobi,  and  PMP  theories  may  result  in  differential  equations  where  an  an¬ 
alytical  solution  may  not  be  discernible.  The  advent  of  the  digital  computer  in  the 
mid-1950’s  enabled  development  of  numerical  methods  to  solve  these  problems. 

At  this  point  the  historical  summary  of  OC  theory  will  be  left,  not  because  there 
are  not  significant  contributions,  but  rather  because  surveying  significant  accomplish¬ 
ments  is  not  the  main  focus  of  this  chapter,  and  in  fact  could  be  the  topic  of  study 
in  itself  taking  significant  time  to  study,  analyze,  and  summarize.  Suffice  it  to  say 
that  efforts  from  the  1950’s  to  present  focus  on  numerical  techniques  to  solve  prob¬ 
lems  that  were  previously  intractable,  analytically  or  numerically,  with  pencil  and 
paper  analysis,  and  on  showing  that  the  numerical  techniques  produce  results  that 
are  complimentary,  if  not  equivalent,  to  CV-based  solutions.  Bellman’s  principle  of 
optimality  and  a  sequential  decision  making  method,  known  as  Dynamic  Program¬ 
ming,  was  developed  in  the  1950’s  and  is  still  widely  used  [16].  Following  dynamic 
programming,  nonlinear  programming,  based  on  gradient  search  methods,  was  devel¬ 
oped  in  the  1960’s  [16],  and  spectral  methods  were  developed  in  the  1970’s  and  more 
prolihcally  applied  to  OC  problems  in  the  1980’s  and  1990’s  [7]. 

This  has  been  a  brief  survey  of  the  history  of  theory  related  to  solving  OC  prob¬ 
lems.  It  is  by  no  means  exhaustive,  and  was  never  intended  to  be.  The  intent  of  this 
brief  survey  was  to  give  a  historical  context  for  methods  that  will  be  discussed  in  this 
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document  for  solving  OC  problems.  In  the  next  section,  the  general  OC  problem  will 
be  stated  and  the  CV-based  formulation  of  the  solution  will  be  presented. 

2.2  Optimal  Control  Problem  Defined 

In  this  section  the  general  continuous-time  OC  problem  is  stated  and  the  general 
procedure  for  solving  the  problem  is  summarized.  Many  textbooks  have  been  written 
on  the  subject  such  as  [54]  and  [17]  and  contain  derivation  of  the  Euler- Lagrange 
(EL)  equations  using  CV  principles.  The  concepts  in  this  section  are  summarized 
from  [54],  [75],  [7],  and  [50].  The  notation  used  is  a  combination  of  that  used  in 
[54],  [72],  [7],  and  [50],  attempting  to  match  the  most  commonly  used  notation  in 
current  literature.  References  [54]  and  [48]  also  present  equivalent  development  of 
discrete-time  Euler- Lagrange  equations,  which  will  not  be  summarized  here. 

2.2.1  Formulation. 

The  objective  of  an  OC  problem  is  to  find  an  admissible  control  function  u *(f)  that 
transfers  a  system  from  some  initial  state  x0  at  the  initial  time  t0  to  some  final  state 
x/  at  the  final  time  tf  following  an  admissible  state  trajectory  x*(f)  that  minimizes 
a  performance  index,  J,  of  the  form: 

J (x(f),  u (t),t)  =  $(x(tf),tf)  +  [  g(x(t),  u (t),t)dt  (2.7) 

Jto 

The  term  admissible  is  used  to  indicate  that  the  optimal  solutions  u*(£)  and  x*(£)  lie 
within  acceptable  upper  and  lower  limits.  The  “*”  superscript  notation  is  commonly 
used  in  many  texts  to  distinguish  the  solution  that  minimizes  the  functional  J.  The 
first  term  in  (2.7),  $(•),  is  known  as  the  Mayer  term  and  the  integral  term,  f**  g(-)dt, 
is  known  as  the  Lagrange  term.  The  cost  functional  is  said  to  be  in  Bolza  form  when 
both  terms  are  present.  The  state  vector,  x(-),  is  in  Mn,  the  control  vector,  u(-),  is  in 
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Mm.  In  vector  form,  x(t)  and  u(i)  can  be  written  as: 


- 1 

_ 1 

1 - 

HO 

,  U  (t)  = 

Xn{t) 

1 

•HO 

The  performance  index  (2.7)  minimization  is  subject  to  the  dynamical  constraints  of 
the  system  to  be  controlled,  generally  represented  as: 


*(t)  =  f(x(t),u(t),t)  (2.8) 

The  boundary  conditions,  </>(•),  in  general  form  can  be  stated  as: 

0(x(to),  to,  x{tf),tf)  =  0  (2.9) 

Some  of  the  boundary  conditions  that  may  be  specified  are  summarized  below. 

•  Fixed  Final  Time  -  The  terminal  time  may  be  specified  with  x(£y)  fixed,  x(£j) 
free,  or  x(£y)  on  a  surface  m(x(t))  =  0. 

•  Free  Final  Time  -  The  terminal  time  may  not  be  specified  with  x(£y)  fixed, 
x(fj)  free,  x(£j)  on  a  moving  point  9{t),  x.(tf)  on  a  surface  m(x(t))  =  0,  or 
x(£j)  on  a  moving  surface  m(x(£),£)  =  0. 

Constraints  may  also  be  placed  on  the  path  the  admissible  solution  can  take,  C(-), 
including  boundaries  on  admissible  x(t)  trajectories,  generally  represented  by  the 
inequality  constraints: 

C(x(£),  u(£),  t)  <  0  (2.10) 


17 


If  the  set  of  admissible  controls  is  limited  to  some  bounded  region  U ,  written  as, 
u  (t)  E  U ,  then  the  inequality  constraints  placed  on  the  control  can  be  written  as: 


u. 


i, mm 


(2.11) 


where  i  =  1  The  functional  mappings  of  $(•),  g(-),  /(•),  0(-),  and  C(-)  in 

equations  (2.7)  -  (2.10)  are  given  by: 


$(x0/)iAf) 

g(x(t),u(t),t) 

/(x(*),u(t),t) 

0(x(to),to,x(t/),fy) 

C(x(t),u(t),t) 


rxi-^R 
Mn  x  Mm  xl-)l 
Mn  x  Mm  x  M  — x  Mn 
Mn  x  R  x  Rn  x  M  -x  M9 
Mn  x  Mm  xl^r 


(2.12) 


A  brief  development  of  the  solution  method  based  on  CV  techniques  is  presented 
next.  Equations  to  apply  boundary  conditions  will  be  presented  for  three  of  the  cases 
listed  above;  tf  specified  (fixed)  with  x(t^)  unspecified  (free),  tf  free  with  x(ty)  fixed, 
and  both  tf  and  x(t^)  fixed.  Equations  needed  to  apply  other  types  of  boundary 
conditions  can  be  found  in  [54], 


2.2.2  Classical  Analytical  Solution. 

With  the  general  OC  problem  defined  in  the  previous  section,  a  general  solu¬ 
tion  procedure  based  on  CV  can  be  obtained.  The  development  that  follows  is  a 
combination  of  information  found  in  [7,50,54,72],  The  necessary  conditions  for  a 
minimizing  solution  are  obtained  by  looking  at  the  first  variation  of  the  functional  in 
(2.7)  augmented  by  the  differential  equality  constraints  (system  dynamics),  inequal¬ 
ity  path  constraints  (C(-)),  and  boundary  conditions  (0(-)),  denoted  as  Ja.  Lagrange 
multipliers  v  E  M9,  A (t)  E  Mn,  referred  to  as  the  costate  vector,  and  fi(t)  E  Mc  are 
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introduced  to  augment  the  cost  functional  with  the  boundary  conditions,  dynamics, 
and  path  constraints  respectively.  The  augmented  cost  functional  is  given  as: 

Ja  =  $(x(t7), tf)  -  uT  0(x(to), t0> 

+  [  (0(x(t),u(t),t)  -  AT(t)  [x(t)  -  f(x(t),u(t),t)]  (2.13) 

Jto 

— ^TC(x(t),  u(i),  £)}d£ 

Taking  the  first  variation  of  Ja,  represented  as  SJa,  following  procedures  similar  to 
those  found  in  [7,50,54]  gives: 

SJa  =  [$x(*/)  -  UT4>x(tf)]6^f  -  [vT(t)x(to)}8x0  -  5uT(j)  +  [$t(tf) 

- vTMtf )  +  9(tf)  -  AT(t/)(k(t/)  -  f {tf))  -  nT(tf)C{tf)]8tf 
+[-uT(f)t(t0 )  -  g(t0)  +  A T(h))(x(h))  -  f(io)  +  Mr(to)C(t0)]^o  (2.14) 
+  ft0f{(9x  +  ATfx  -  firC X)hx  +  (,gu  +  ATfu  -  /atCu)5u 
— <5AT(x  -  f)  -  5vt C  -  AT<5x}dt 

Notice  that  the  arguments  of  the  functions  in  (2.14)  were  left  out  for  readability.  It’s 
clear  from  (2.13)  what  the  functional  dependencies  are.  Next,  define  the  Hamiltonian 
functional: 

(t),tM(t),t)  =  0(x(t),u(t),t)  +  AT(t)f(x(t),u(t),t)  (2.15) 


19 


Substituting  (2.15)  along  with  the  partial  derivatives  7/x,  "Ha,  and  "Hu  into  (2.14), 
using  the  following  integration  by  parts  formula  on  the  6±  term: 


A 1 5k  dt  =  —A T(tf)5x(tf)  +  A7  (t0)Sx(t0)  + 


A  <5x  dt  (2.16) 


incorporating  the  following  identities  for  <5x/  and  hxj: 


=  Sx(tf)  +  ±(tf)Stf 
hx0  =  5x(t0)  +  x(t0)5t0 


(2.17) 


and  simplifying  yields: 


dJa  =  [$x(t/)  -  -  AT(t/)]<5x/  +  [-^^(to)  +  A7  (t0)]5x0 

-dvT(\>  +  ~  vT(j)t{tf)}8tf  +  {-U{t0)  -  vT4>t(tG)}  (2.18) 

Sto  +  J%{[nx  +  Ar]hx  +  nu5u  +  (f  -  x)hAT  -  C diiT}dt 


The  EL  equations  needed  to  solve  for  the  OC  and  the  associated  optimal  state 
trajectory  come  from  the  integrand  of  (2.18).  The  variational  terms  inside  the  inte¬ 
grand  are  arbitrary,  thus  the  necessary  conditions  for  a  minimum  are  found  by  finding 
the  coefficients  that  cause  the  variations  hx,  hu,  and  5 A  to  vanish.  Therefore,  the 
necessary  conditions  that  must  be  satisfied  for  a  minimizing  solution,  regardless  of 
the  boundary  conditions,  are: 

•  State  equations:  The  6 A  term  becomes  zero  if  (f  —  x)  =  0.  Therefore,  the 
state  equations  are  found  by  taking  the  partial  derivative  H\  of  the  Hamilto¬ 
nian.  This  may  seem  a  little  redundant  since  the  state  equations  are  given  as 
differential  equality  constraints  in  (2.8),  but  confirms  that  the  Hamiltonian  has 
been  constructed  correctly  and  that  the  minimization  depends  on  the  system 


20 


dynamics. 

•  T 

•  Costate  equations:  The  5x  term  becomes  zero  if  ("Hx  +  A  )  =  0.  This 
condition  defines  a  set  of  differential  equations  for  the  costates  by  taking  the 
negative  of  the  partial  derivative  T~LX  of  the  Hamiltonian. 

•  Control  equations:  The  <5u  term  becomes  zero  if  the  partial  derivative  'Hu  is 
zero.  This  generally  produces  an  OC  solution  that  is  a  function  of  the  costates 
and  possibly  the  states.  This  condition  is  true  for  an  unbounded  control.  Pon- 
tryagin’s  Minimum  Principle  (PMP)  will  be  discussed  below  to  account  for 
bounded  controls. 

The  variational  terms  in  (2.18)  that  are  not  inside  the  integral  operator  define 
how  boundary  conditions  are  applied.  Generally  speaking,  if  a  boundary  condition  is 
specified,  either  initial,  final,  or  both,  then  the  variation  related  to  that  quantity  is 
zero  and  the  term  drops  out  of  equation  (2.18).  For  example,  if  the  initial  time  and 
initial  state  are  given,  then  5t0  and  <5x0  are  0.  If  a  quantity  is  unspecified,  or  free, 
then  the  variation  on  that  quantity  is  arbitrary  and  the  minimizing  solution  must 
drive  the  coefficient  on  that  term  to  zero.  For  example,  if  the  final  state  is  free,  then 
<5x/  is  arbitrary,  thus  [$x(t/)  —  v7  fi>x(tf)  —  X7  (f/)]  must  be  0.  This  reasoning  allows 
for  the  definition  of  three  common  boundary  conditions:  fixed  final  time  and  free  final 
state ,  fixed  final  state  and  free  final  time,  and  fixed  final  state  and  fixed  final  time  as 
follows. 

•  Fixed  final  time  with  free  final  state:  In  this  case,  the  coefficient  on  the 
<5x/  term  must  be  zero.  Therefore: 

XT(tf)  =  $x(tf)  -  iyT<t>x(tf)  (2.19) 

•  Fixed  final  state  with  free  final  time:  In  this  case,  the  coefficient  on  the 
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5tf  term  must  be  zero,  leading  to: 


®t(tf)-vT<l>t(tf)+g(tf)-  fxT(tf)C(tf)  +  XT(tf){(tf)  =0  (2.20) 

Mtf)  ~  +  Hitf)  =  0 

•  Fixed  final  state  with  fixed  final  time:  This  is  the  simplest  case  where 
x(i0)  =  x0  and  x(£f)  =  xj  provides  the  necessary  information  to  determine 
the  solution  by  treating  the  system  of  equations  as  a  Boundary  Value  Problem 
(BVP). 

Other  boundary  conditions  may  be  specified  as  previously  mentioned.  A  table  of  the 
relevant  boundary  condition  equations  is  found  in  [54], 

Looking  at  the  first  variation  of  Ja  has  revealed  a  procedure  for  finding  necessary 
conditions  for  a  minimum  solution.  The  variational  method  effectively  translates  to 
finding  the  u*(i)  and  the  associated  x*(i)  that  minimizes  the  cost  functional  of  (2.7) 
to  finding  the  u*(t)  and  the  associated  x*(i)  that  minimizes  the  Hamiltonian  in  (2.15). 
The  process  can  be  stated  as: 

1.  Build  the  Hamiltonian  using  (2.15) 

2.  Write  the  state  equations  as: 

PfU 

x*  =  |(xt(t))ut(t),V(t),/it(t)It)  (2.21) 

3.  Define  the  costate  equations  as: 

fYU 

A*  =  -^(x*(t),u*(t),  A*(t),/**(t),t)  (2.22) 
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4.  Express  the  control  equations  as: 

o-iv 

0  =  —  (x*(t),u*(t),A*(t),/x*(t),t)  (2.23) 

5.  Simultaneously  solve  the  system  of  Euler- Lagrange  equations  (2.21),  (2.22),  and 
(2.23)  and  apply  the  appropriate  boundary  condition  equations  from  [54].  The 
boundary  condition  equations  determine  the  terminal  time  and  state,  resulting 
in  what  is  commonly  referred  to  as  the  Hamiltonian  Boundary  Value  Problem 
(HBVP).  Note  that  (2.23)  defines  u*(i)  as  a  function  A *(£),  /z*(i),  and  x*(£) 
which  may  be  substituted  into  (2.21)  and  (2.22)  to  remove  explicit  functionality 
of  u *(t)  if  u*(i)  can  be  isolated. 

The  u*(£)  determined  by  (2.23)  is  a  necessary  condition  for  an  optimum  solution 
in  the  case  where  there  are  no  bounds  on  the  control  or  where  the  resultant  u*(f) 
never  breaks  the  upper  or  lower  limits  on  u(t).  The  necessary  and  sufficient  conditions 
for  "H(x*(t),  u*(t),  A*(t),  t)  to  be  a  global  minimum  are  that  equations  (2.21), 
(2.22),  and  (2.23)  be  satisfied  ( necessary )  and  that  the  m  x  m  matrix  "Huu  be  positive 
definite  ( sufficient )  [75].  In  order  to  pick  an  admissible  u*(£),  it’s  necessary  to  consider 
how  to  account  for  the  bounded  control  in  (2.11)  using  PMP.  The  necessary  condition 
provided  by  PMP  states  that  for  all  admissible  controls  u{t)  G  U: 

U (x* (t ) ,  u* (t) ,\*(t),n*(t),t)  <  H (x* (t) ,  n(t ) ,  A* (t),  n*(t),t)  (2 .24) 

This  equation  says  that  if  there  is  an  admissible  OC  signal  u*(£)  then  there  exists  an 
optimal  costate  vector  A*  (t)  and  vector  of  Lagrange  multipliers  /x*  (t)  multiplying  the 
path  constraints  that  satisfies  (2.22)  and  (2.10)  at  every  point  in  the  time  interval 
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t  E  [to,tf]  such  that  (2.24)  is  true  [75].  That  is  to  say  that: 


min 'H (x* (t) ,  u(t) ,  A* (t) ,  n* (t) ,t)  =  U (x* (t) ,  U* (t) ,  A* (t) ,(J.*(t),t)  (2.25) 

uGi/ 

The  OC  is  the  one  out  of  the  entire  set  of  admissible  controls  that  causes  1~L  to  be 
its  global  minimum.  In  practice,  applying  the  PMP  involves  solving  for  x*(t),  A *(t), 
and  u *{t)  using  equations  (2.21)  -  (2.23),  initially  assuming  unbounded  controls,  and 
then  comparing  the  resultant  u*  with  the  admissible  region  to  see  if  there  are  areas 
where  u*  is  out  of  bounds.  If  so,  then  the  control  signal  is  re-evaluated  using  (2.25) 
until  the  minimum  H  is  found  that  satisfies  the  given  boundary  conditions  [54], 

An  alternate  method  of  taking  into  account  boundaries  on  the  state  trajectory  is 
presented  in  [54],  which  may  eliminate  (2.10)  from  the  problem  (if  C(-)  only  consists 
of  bounds  on  the  states)  without  adding  significant  complexity.  Kirk  presents  a  pro¬ 
cedure  for  transforming  the  c  inequality  constraints  into  a  single  equality  constraint. 
The  net  effect  is  that  there  are  two  additional  equations,  one  for  xn+\  and  one  for 
An+ 1 .  Therefore,  there  are  n  +  1  equations  in  (2.21)  and  in  (2.22)  to  solve.  The 
procedure  stays  virtually  the  same  as  indicated  above,  but  eliminates  the  Lagrange 
multipliers  /x,  and  does  not  affect  applying  the  PMP  for  determining  the  admissible 
control  vector. 

In  general,  analytically  solving  the  system  of  differential  equations  (2.21)  and 
(2.22)  with  substitutions  made  using  (2.23)  is  only  possible  for  very  simple  problems. 
These  equations  are  typically  coupled  nonlinear  systems  where  standard  integral  ta¬ 
bles  may  not  offer  closed-form  solutions  and  linear  methods  such  as  Laplace  trans¬ 
forms  are  not  helpful.  Therefore,  numerical  methods  are  necessary  to  find  the  solution 
to  most  meaningful  OC  problems. 
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2.2.3  Formulation  and  Classical  Solution  Summary. 


In  this  section,  the  OC  problem  was  defined  and  the  Calculus  of  Variations  and 
Pontryagin’s  Minimum  Principle  were  used  to  derive  the  Hamiltonian  Boundary  Value 
Problem,  which  defines  the  first-order  necessary  conditions  for  optimality.  Solving 
the  resulting  set  of  differential  equations  indirectly  solves  the  original  OC  problem  by 
finding  the  solution  that  minimizes  the  Hamiltonian,  which  turns  out  to  also  be  the 
minimizing  solution  to  the  original  cost  functional.  Solving  the  differential  equations 
of  the  HBVP  analytically  can  be  difficult,  if  not  impossible,  requiring  numerical  tech¬ 
niques  to  solve  the  problem.  A  survey  of  numerical  techniques  for  solving  the  OC 
problem  are  described  in  the  next  section. 

2.3  Numerical  Methods  for  Optimal  Control 

There  are  two  main  categories  of  numerical  approaches  for  solving  the  OC  prob¬ 
lem  defined  by  (2.7)  -  (2.11):  indirect  methods  and  direct  methods.  Indirect  meth¬ 
ods  are  based  on  numerically  solving  the  system  of  Ordinary  Differential  Equations 
(ODEs),  known  as  the  Hamiltonian  Boundary  Value  Problem,  derived  by  the  CV 
approach  described  in  section  2.2.2  by  equations  (2.21)  -  (2.25).  Direct  methods,  on 
the  other  hand,  translate  the  continuous-time  OC  problem  into  a  nonlinear  optimiza¬ 
tion  problem,  also  called  a  Nonlinear  Programming  (NLP)  problem,  where  gradient 
search  methods  are  applied  to  determine  the  optimal  state  trajectories  and  control 
sequences  by  satisfying  Karush-Kuhn- Tucker  (KKT)  optimality  conditions  [71].  This 
section  will  briefly  describe  typical  methods  used  in  both  categories  and  then  discuss 
PSMs  used  for  solving  OC  problems. 
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2.3.1  Indirect  Methods. 


Indirect  methods  are  based  on  solving  the  HBVP  given  in  equations  (2.21)  - 
(2.23)  subject  to  the  given  boundary  conditions.  The  benefit  of  using  the  CV-based 
indirect  approach  is  that  it  yields  highly  accurate  results  while  providing  assurances 
that  the  first-order  optimality  conditions  are  satisfied  [50] .  However,  difficulties  arise 
in  that  explicit  derivations  of  the  costate  (2.22)  and  control  (2.23)  equations  are 
required,  which  can  be  very  difficult  depending  on  the  OC  problem  being  considered, 
and  prior  knowledge  of  the  activeness  of  inequality  constraints  (2.10)  is  necessary 
[78].  Numerical  techniques  applied  to  the  indirect  equations  generally  require  good 
initial  guesses  of  the  costates,  which  is  often  difficult  since  the  costates  generally 
do  not  have  direct  physical  interpretations  [7,  50] .  There  are  two  main  approaches 
to  solving  the  problem:  shooting  and  collocation  methods.  Shooting  methods  are 
iterative  techniques  that  enable  solution  of  the  BVP  using  Initial  Value  Problem 
(IVP)  techniques  while  collocation  methods  allow  simultaneous  solution  of  unknown 
parameters  [71].  Consider  the  general  problem  of  finding  the  solution  x{t ),  where 
t  G  [to,tf],  to  an  IVP  of  the  form: 

jtx(t)  =  f(x(t),t)  (2.26) 

x(t0)  =  x0 

Equation  (2.26)  is  presented  here  as  a  single  ODE,  but  can  in  general  represent  a 
system  of  ODEs,  which  will  be  the  case  when  discussing  shooting  methods.  The  solu¬ 
tion  to  the  IVP  involves  breaking  [to,tf]  into  n  subintervals,  where  the  ith  subinterval 
is  [U,ti+ 1]  with  length  hi  =  tl+\  —  ti.  In  many  applications,  the  lengths  of  the  n 
subintervals  are  equal  and  the  size  is  denoted  as  h ,  but  it  is  not  necessary  that  the 
subinterval  lengths  be  equal.  The  objective  is  to  find  the  solution  at  each  ith  node  as 
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given  in  [71],  which  can  be  written  as: 


Xi- (-1  Xi  “f 


rti+ 1 


f(x(t),t)dt 


(2.27) 


The  techniques  in  this  section  are  designed  to  estimate  the  integral  in  (2.27).  The 
subsections  below  outline  time-marching  algorithms  to  solve  the  IVP  with  extension 
to  shooting  methods  to  solve  a  BVP  and  collocation  techniques  to  solve  the  BVP. 


2. 3. 1.1  Time-Marching  and  Shooting  Methods. 

A  time-marching  technique  solves  the  IVP  by  using  the  solution  of  xt  (x(t))  and 
possibly  more  previous  points  to  determine  the  subsequent  xi+i  (x(t  +  h))  solution. 
There  are  many  ODE  IVP  solution  techniques,  but  again,  OC  problems  are  BVPs 
or  Multi-Point  Boundary  Value  Problems  (MPBVP).  These  ODE  solution  techniques 
still  apply  when  implemented  in  a  shooting  algorithm  that  enables  terminal  boundary 
conditions  to  be  satisfied.  Some  of  the  most  common  solution  methods  for  IVPs  are 
summarized  here  followed  by  a  brief  description  of  shooting  methods. 

There  are  several  first-order  approximations  available  to  solve  the  IVP.  The  For¬ 
ward  Euler  (FE),  Backward  Euler  (BE),  Leap  Frog  (LF),  and  Trapezoidal  Method 
(TM)  given  in  [12, 67]  are  examples  where  difference  formulas  are  used  to  discretize 
the  time  derivative  on  the  left-hand-side  of  (2.26)  resulting  in  time-marching  rules 
given  in  Table  2.1.  The  Order  of  Error  column  indicates  the  magnitude  of  error  in 


Table  2.1.  Simple  numerical  schemes  for  time-marching  [12,67] 


Method 

Equation 

Order  of  Error 

FE 

%i+ 1  =  Xi  +  hf{xi,ti) 

0{K2) 

BE 

Xi-\- 1  Xi  T  h/(^j+l ,  t j+l) 

0{h 2) 

LF 

xi+i  =  Xi- 1  +  2  hf(xi,ti) 

0(h3) 

TM 

Xi+l  =  Xi  +  |(/(xi+i,fi+i)  +  f(Xi,ti )) 

0{h3) 
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the  estimate  of  x,+i  based  on  the  truncated  Taylor-series.  For  example,  if  h  is  0.1, 
then  for  the  FE  and  BE  the  estimate  errors  will  be  on  the  order  of  10”2  while  the 
LF  and  TM  will  have  errors  on  the  order  of  10-3.  Also  note  that  the  BE  and  TM 
methods  are  implicit  methods  because  the  functions  on  the  right-hand-sides  of  the 
respective  schemes  are  evaluated  at  the  point  being  solved  for  on  the  left-hand-side. 
An  explicit  scheme  is  necessary  to  estimate  the  solution  Xi+\  which  is  then  used  in  the 
right-hand-side  of  an  implicit  scheme.  The  implicit  calculation  of  Xi+\  then  refines 
the  solution  generated  by  the  explicit  equation  [9,67].  These  methods  are  reasonably 
simple  to  implement,  but  can  be  computationally  expensive  since  the  step  size  must 
be  very  small  to  accurately  approximate  derivatives  using  piecewise  linear  functions. 
These  methods  also  have  stability  limitations  that  can  affect  the  solution  if  the  step 
size  is  not  properly  chosen  [67,71].  An  improved  second-order  Euler  method  (error 
0{h3))  is  presented  in  [12]  as: 

xi+i  =Xi  +  |  f(xi ,  U)  +  ^/(xi  +  hf(xi ,  ti),  ti+i)  (2.28) 

While  this  implicit  scheme  may  provide  more  accurate  estimates,  there  are  generally 
more  effective  time-marching  algorithms  that  are  preferred.  The  Taylor-series  method 
can  be  used  to  generate  a  scheme  of  potentially  high  accuracy  [53].  A  fourth-order 
approximation  (error  0{h5)),  assuming  uniform  time  step,  is  achieved  by  writing  out 
the  Taylor-series  for  xi+1  as: 

%i+ 1  =  Xi  +  hx'  {ti)  +  ^x"{U)  +  -yx''\U)  +  ^j-x(4)(fj)  +  0{h5)  (2.29) 

To  use  this  equation  it  is  necessary  that  the  partial  derivatives  with  respect  to  time 
of  f(x,t)  exist  up  to  the  highest  order  derivative  in  the  Taylor-series  expansion  cho¬ 
sen,  fourth  derivative  in  this  case.  The  difficulty  with  this  method  is  that  analyt- 


ical  determination  of  the  derivatives  may  become  very  difficult  depending  on  the 
composition  of  f(x(t),t).  Fortunately,  symbolic  software  manipulation  packages  like 
Mathematica™and  MATLAB®can  help  with  the  differentiation. 


A  more  efficient  method  is  the  commonly  used  Runge-Kutta  (R-K)  scheme.  The 
classical  fourth-order  (error  0(h5))  R-K  scheme  is  discussed  in  [9,12,19,53,65]  and 
written  as: 


K  i  =  ) 


(2.30) 


K,i  =  f(xi  +  hK3,ti+i) 

%i+ 1  =  Xi  +  7r(Al  +  2A"2  +  2K3  +  A4) 
6 


Several  modifications  have  been  made  to  the  classical  R-K  scheme  to  introduce  an 


adaptive  step  size.  Two  notable  schemes,  Runge-Kutta-Fehlberg  [53]  and  Runge- 
Kutta- Dormand- Prince  algorithms,  evaluate  fourth-  and  fifth-order  R-K  estimates 
and  use  the  difference  to  determine  the  estimate  error,  which  is  then  used  to  determine 
the  step  size  for  the  next  iteration  [19,24].  The  Dormand-Prince  method  is  used  in 
the  Matlab® ode45  ODE  solver. 

The  methods  used  to  solve  the  IVP  thus  far,  Euler  methods,  LF,  TM,  Taylor- 
series,  and  R-K,  are  single-step  methods,  meaning  that  only  data  from  the  previous 
mesh  point  is  needed  to  approximate  x,+i  [12].  The  R-K  methods  are  known  as 
multiple- stage  methods  since  they  evaluate  f(x,t )  at  multiple  places  in  [£*,  ii+i]  to 
calculate  xi+i  as  a  weighted  sum  [71].  Multiple-step  algorithms  are  designed  to  be 
more  efficient  by  using  function  evaluations  from  more  than  one  previous  time  point. 
Two  of  the  most  common  methods  are  the  explicit  Adams-Bashforth  (A-B)  and  im- 
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plicit  Adams- Moulton  (A-M)  schemes  [9,71].  Fifth-order  A-B  and  A-M  schemes  are 
given  respectively  as: 


xi+i  =  Xi  +  ^[1901/(xi,ti)  -  2774/(xi_i,ti_i)  +  (2.31) 

2616/(xj_2,ti-2)  -  1274/ (xi-3,  tj_3)  +  251/(x;_4,  t^4)\ 

xi+1  =  Xi  +  ^[251f(xi+1,ti+1)  +  646  f(xi,ti)  +  (2.32) 

— 264/(xj_i,  ti-i)  +  106/(xi_2,ii-2)  -  19/(xi-3,ii-3)] 

The  A-B  and  A-M  schemes  are  often  used  together  as  a  predictor- corrector  scheme 
where  the  A-B  (2.31)  is  used  to  predict  the  value  of  Xj+i  which  is  then  corrected  by 
the  A-M  (2.32),  improving  the  quality  of  the  solution  approximation  [12,53].  Since 
the  A-B  and  A-M  schemes  require  either  three  or  four  previous  function  evaluations, 
any  of  the  single-step  methods,  R-K  for  example,  can  be  used  to  start  the  process 
until  sufficient  data  is  available  for  the  A-B/A-M  predictor-corrector  to  take  over  [12]. 

Now  that  a  sampling  of  the  most  common  time-marching  methods  for  solving 
IVPs  has  been  presented,  shooting  methods  can  be  described  that  allow  the  IVP 
solution  schemes  to  be  used  to  solve  the  BVP  [49,71].  Consider  the  following  ODE 
to  be  solved  on  the  interval  t  6  [to,tf]  with  given  boundary  conditions  Xq  and  Xf\ 


d 2  , 
drxlf) 

x(t0) 

=  X0 

(2.33) 

x(tf) 

=  xf 

This  ODE  can  be  written  as  a  first-order  system  of  ODEs  by  changing  variables. 
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Letting  X\  —  x  and  4rX\  =  x2  leads  to  the  following  system  of  first-order  ODEs  that 
can  be  solved  by  any  of  the  previously  described  time-marching  techniques. 


=  x2(t)  (2.34) 

=  f{t,Xi{t),X2{t)) 

The  simple  shooting  method  converts  the  BVP  in  (2.33)  to  an  IVP  in  (2.34)  by  setting 
the  initial  conditions  as: 


£l(f0) 
X2  (t0) 


x(t0)  =  x0 
d  ,  . 
jtxM  =  “ 


(2.35) 


An  error  function  is  necessary  to  refine  guesses  of  u.  The  error  function,  E(u),  can 
be  defined  as: 

E(u)  —  x(tf;u)  —  Xf  (2.36) 

Equation  (2.36)  basically  describes  the  shooting  method  as  a  root-finding  problem 
where  the  objective  is  to  find  u  such  that  E(u)  is  driven  to  zero  [71].  Thus,  the 
procedure  is  as  follows: 

1.  Set  initial  guess  of  u  in  (2.35) 

2.  Apply  time-marching  scheme  of  choice  to  integrate  (2.34)  from  t0  to  tf 

3.  Calculate  E(u)  using  (2.36):  The  solution  has  been  found  if  the  error  is  within 
a  desired  tolerance  (e),  \E(u)\  <  e,  else 

4.  Update  guess  of  u  using  root-finding  scheme.  Newton’s  method,  modified  New¬ 
ton’s  method,  and  secant  method  are  some  of  the  most  common  root-finding 
algorithms. 
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5.  Repeat  process  until  \E(u)\  <  e  condition  is  achieved. 

The  simple  shooting  method  is  attractive  to  use  because  of  its  simplicity.  However, 
there  are  numerical  shortcomings  of  this  method,  such  as  slow  convergence  [49]  when 
the  guess  of  u  is  far  from  the  root  of  (2.36),  assuming  that  a  globally  convergent  New¬ 
ton  method  is  being  used  that  is  guaranteed  to  find  the  root  [53],  or  non-convergence 
when  applied  to  the  OC  problem  due  to  “ill  conditioning  of  the  Hamiltonian  dy¬ 
namics”  [71].  A  multiple-shooting  method  can  be  used  to  alleviate  these  numerical 
issues. 

The  multiple  shooting  method  is  based  on  the  problem  formulation  given  in  (2.34)  - 
(2.36).  The  difference  is  that  the  time  interval  [to,  £/]  is  broken  into  M+l  subintervals, 
where  M  is  the  number  of  interior  mesh  points,  and  shooting  is  applied  over  each 
[ti,  ti+\]j,  j  =  1, ...,  M  +  1,  subinterval.  The  values  of  x\  and  X2  are  unknowns  in  each 
subinterval  to  be  determined  through  time-marching  and  root-finding.  Continuity 
constraints  are  placed  on  adjoining  j  and  j  +  1  subinterval  boundaries  by: 


Xl,i+l 

j 

x2,i+l 


_  TJ+1 
A 


_  TJ+1 

x2,i 


0 

0 


(2.37) 


Therefore,  the  root-finding  problem  is  to  find  the  U{+±  slopes  that  satisfy  the  continuity 
conditions  and  the  given  boundary  conditions  [49,53,71]. 

The  advantage  of  the  multiple  shooting  method  is  that  it  is  very  flexible  in  the 
types  of  constraints  considered  in  OC  problems,  such  as  bounded  states,  controls, 
and  path  constraints  [78].  Very  accurate  results  can  be  achieved  while  alleviating  the 
aforementioned  numerical  issues,  but  it  is  cumbersome  to  implement,  considerable 
analytical  work  may  be  required  in  deriving  the  costate  equations,  a  good  initial 
guess  of  the  solution  is  necessary,  and  may  introduce  discontinuities  in  a  continuous 
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problem  that  can  not  be  resolved  [49,78].  Holsapple,  Venkataraman,  and  Doman 
developed  a  modified  simple  shooting  method  in  [49]  that  combines  the  “favorable 
aspects”  of  both  the  simple  and  multiple  shooting  methods  resulting  in  a  “superior, 
faster  method  for  solving  TPBVPs.”  This  research  shows  that  improving  shooting 
methods  is  still  an  active  research  topic  in  the  OC  community. 

The  shooting  methods  described  in  this  section  can  be  applied  to  the  indirect 
CV-based  OC  problem  formulated  by  (2.21)  -  (2.23),  according  to  [71].  Guesses  are 
made  for  the  unknown  initial  conditions,  time  marching  applied  to  solve  the  system 
of  ODEs,  and  error  evaluation  performed  to  check  satisfaction  of  boundary  conditions 
at  the  end  of  the  time  interval.  Refinements  to  guesses  of  the  unknown  conditions 
are  made  through  application  of  root-finding  techniques  and  iterations  made  until  the 
solution  is  found  that  satisfies  terminal  constraints.  The  procedure  is  essentially  the 
same  for  both  simple  and  multiple  shooting  methods  with  the  exception  that  multiple- 
shooting  applies  the  algorithm  over  a  specified  number  of  subintervals  introducing 
more  variables  to  be  solved. 

2.3. 1.2  Indirect  Collocation. 

Collocation  methods  have  become  quite  popular  in  the  OC  community  over  the 
past  couple  of  decades.  They  are  very  powerful  techniques  for  solving  the  HBVP  given 
in  (2.21)  -  (2.23).  These  methods  are  also  very  powerful  in  solving  the  OC  problem 
using  direct  methods ,  discussed  in  section  2.3.2.  Collocation  techniques  form  the 
basis  for  the  PSM,  discussed  in  section  2.3.3,  that  will  be  used  in  this  research.  Rao 
presents  a  brief  description  of  collocation  methods  in  [71]  with  the  general  idea  being 
that  the  unknown  solution  is  approximated  using  a  polynomial,  or  set  of  polynomials. 
Unknown  coefficients  in  the  polynomials  are  simultaneously  solved  for  using  root¬ 
finding  techniques  such  that  estimates  of  the  states  and  derivatives  of  the  states 
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match  the  analytic  function  at  a  set  of  discrete  points. 
Again,  consider  the  general  BVP  of  the  form: 


d  ,  . 

itx[t) 

=  f(x(t),t) 

x(t0) 

=  Xo 

(2.38) 

z(tf) 

=  xf 

This  ODE  can  be  a  single  function  or  a  system  of  functions,  which  is  generally  con¬ 
sistent  with  the  EL  state  and  costate  equations.  Also  suppose  that  N  discrete  time 
points  are  chosen  in  [£0,£/],  called  collocation  points  and  denoted  as  t%  for  i  —  1,  ...N, 
resulting  in  N  + 1  subintervals.  The  points  can  be  chosen  in  various  ways,  and  again, 
the  distance  between  two  mesh  points  rt  and  t1+\  for  i  =  0,...,N  —  1  need  not  be 
uniform.  In  fact,  the  mesh  points  used  in  the  PSM  are  not  uniformly  spaced.  Ad¬ 
ditionally,  depending  on  the  method  used  to  choose  the  collocation  points,  t0  and  tf 
may  or  may  not  be  collocation  points.  For  now,  assume  collocation  points  have  been 
specified.  Options  for  choosing  points  will  be  discussed  later  in  this  section. 

One  collocation  method  is  to  use  a  piecewise  polynomial,  Qk ,  of  degree  K,  to 
approximate  the  solution  x(t )  in  each  subinterval  [U,ti+ 1]  as  described  in  [71],  which 
is  essentially  a  Finite  Element  Method  (FEM)  [14].  The  form  of  the  solution  is: 

K 

x{t)  «  QK(t )  =  ^2ak(t-  U)k,  t  e  [ U , ti+ 1]  =  U  (2.39) 

k= 0 

The  coefficients  (do,  ••.,  cik)  are  unknowns  to  be  solved  for  such  that  the  approximation 
Q{t)  matches  x(t)  at  the  collocation  points,  i.e.  Q{ji)  =  x(r,),  in  the  interval  Jj. 
Furthermore,  the  derivative,  Q(t),  found  by  differentiating  (2.39),  is  set  to  match 
f(x(t),t)  at  the  collocation  points,  known  as  the  collocation  condition.  In  order  to 
find  the  unknown  coefficients,  a  defect  function,  R ,  also  referred  to  in  the  literature 
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as  the  remainder,  residual,  or  error  function,  is  defined  as: 


Rj  =  Q(Tj)  ~  f(x(Tj),Tj ),  for  3  =  1,  -iN 


(2.40) 


The  defects,  which  are  functions  of  the  unknown  coefficients,  can  be  put  into  a  matrix, 


Z, 


(2.41) 


Rn 

and  the  coefficients  can  be  found  simultaneously  by  applying  a  root-finding  scheme 
to  solve  Z  =  0. 

Another,  more  effective,  collocation  technique  involves  using  global  orthogonal 
polynomial  basis  functions  over  the  entire  time  interval.  Instead  of  Qx(t)  defined  by 
(2.39),  a  polynomial  approximation  to  the  unknown  solution  x(t),  as  given  in  [33], 
can  be  written  as: 

N 

x(t)  «  QK(t)  =  ^ak(pk(t)  (2.42) 

k= 0 


where  0*.  is  an  orthogonal  polynomial  basis  function.  In  the  context  of  the  OC  solution 
using  the  indirect  CV-based  formulation,  the  state  (2.21),  costate  (2.22),  and  control 
(2.23)  (if  not  able  to  be  isolated  and  substituted  into  the  state  and  costate  equations) 
equations  can  be  represented  in  this  manner.  The  collocation  points  are  chosen  to  be 
the  roots  of  an  orthogonal  polynomial  of  a  given  order  [71].  There  are  a  variety  of 
orthogonal  polynomial  basis  functions  that  can  be  used  such  as  Legendre,  Chebyshev, 
and  Hermite  polynomials  [29,68,71].  Lagrange  interpolating  polynomials  can  also 
be  used  instead  of  the  orthogonal  polynomials  [14].  The  most  common  collocation 
methods  used  to  solve  OC  problems  are  based  on  Lagrange  interpolating  polynomial 
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bases,  Li.  Letting  <f>k  in  (2.42)  be  a  Lagrange  interpolating  polynomial  gives: 

N 

Mr)  =  Lk(r)  =  TT  (fc  =  0 (2.43) 

?=o r*  -  T> 
i/* 

The  advantage  of  using  Lagrange  polynomials  in  (2.42)  is  that  they  have  the  isolation 
property,  meaning: 

{1  if  k  =  j 

(2,44) 

0  if  k  f  j 

which  leads  to  coefficients,  a*,,  in  (2.42)  being  the  values  of  the  state  at  the  collocation 
points,  i.e.  =  x{jk)  [71]. 

The  most  common  collocation  point  sets  are  Legendre- Gauss  (LG),  Legendre- 
Gauss-Radau  (LGR),  and  Legendre- Gauss-Lobatto  (LGL)  points  [28,33,37,71].  Chebyshev- 
Gauss  (CG),  Chebyshev-Gauss-Radau  (CGR),  and  Chebyshev-Gauss-Lobatto  (CGL) 
points  can  also  be  used  [33].  These  point  sets  are  based  on  Gaussian  quadrature  rules, 
are  the  roots  of  a  particular  polynomial,  and  are  defined  on  the  open,  semi-open,  or 
closed  interval  of  -1  to  1.  Therefore,  the  time  domain  [to,tf]  given  in  the  problem 
must  be  linearly  transformed  into  the  -1  to  1  domain,  and  transformed  back  once  a 
solution  has  been  found.  Fornberg  gives  a  table  in  [33]  with  the  details  of  these  node 
sets,  which  is  summarized  in  Table  2.2. 

The  table  shows  that  the  LG,  LGR,  and  LGL  nodal  sets  are  the  roots  of  Leg¬ 
endre  polynomials  (P)  of  the  degree  specified  by  the  subscript  and  the  CG,  CGR, 
and  CGL  are  the  roots  of  Chebyshev  polynomials  (T)  defined  by  the  recursion  re¬ 
lationships  given.  The  last  column  of  the  table  gives  formulas  for  calculating  the 
weights  associated  with  each  point  set  to  be  used  in  Gaussian  quadrature  (numerical 
integration).  These  weights,  Wk,  will  be  used  in  the  direct  collocation  method  in  the 
PSM.  Numerous  MATLAB®routines  have  been  written  and  are  available  for  download 


36 


Table  2.2.  Collocation  points  and  Gaussian  quadrature  weights  [33] 


Point  Set 

Domain 

Nodes  (xk) 

{k  =  0, ...,  N) 

Weights  (wk) 

{k  =  0, ...,  N) 

LG 

(-1,1) 

Roots  of  PN+ 1 

2 

0--x'i)\pN+i  Pfc)]2 

LGR 

1 

o 

i-S 

T 

Roots  of  P/v  +  PN+i 

W°  —  (7V+1)2 

7/;,  —  1  l~xk 

Wk  (N+ l)2  [PN{xk)Y 

LGL 

[-Mi 

-1,  Roots  of  P'N,  1 

2  2 

wk  -  iV(JV+l)  [PjvRfc)]2 

CG 

(-1,1) 

cos  [ (2fc+1)7r 

[  2N+2 

wk  ~  N+i 

CGR 

i 

o 

i-S 

T 

COS  [2N+l\ 

W°  —  2V+1 

Wk  2N+2 

CGL 

[-1,1] 

cos  [# ] 

W0  =  WN  =  ^ 

Wk=  § 

TM 

front  the  MathWorks  website  to  automatically  calculate  the  collocation  points  and 
quadrature  weights  given  the  number  of  desired  points. 

Solving  the  BVP  in  (2.38)  using  orthogonal  collocation  is  the  same  as  previously 
stated.  The  coefficients  (do,  •••,  clk)  are  unknowns,  the  polynomial  approximation  Q(t) 
and  its  derivative  are  set  to  match  x(t)  and  f(x(t),t)  respectively  at  the  collocation 
points.  The  defect  matrix  defined  by  (2.40)  and  (2.41)  is  constructed  to  simultane¬ 
ously  solve  for  the  unknown  coefficients  using  root-finding  to  solve  Z  =  0. 

Solving  the  HBVP  given  in  (2.21)  -  (2.23)  is  a  natural  extension  of  the  process 
described  here.  Each  of  the  states,  costates,  and  controls  making  up  the  state  vector 
x(i),  costate  vector  A (£),  and  control  vector  u(£)  are  approximated  using  a  series  of 
Lagrange  polynomials  as  given  in  (2.42)  and  (2.43),  the  coefficients  of  which  are  the 
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unknowns  to  be  found.  A  set  of  collocation  nodes  is  chosen  from  Table  2.2,  the  defect 
matrix  is  constructed,  and  nonlinear  root-finding  method  is  applied  as  stated  above. 
Fahroo  and  Ross  demonstrated  this  method  by  using  Legendre  polynomial  approxi¬ 
mations  for  x,  u,  and  A,  collocated  at  LGL  points  to  solve  a  sample  problem  in  [27], 
concluding  that  the  global  orthogonal  collocation  method  is  an  effective  alternative 
to  multiple-shooting  techniques  to  solve  the  HBVP. 

2.3.2  Direct  Methods. 

Indirect  methods  transform  the  continuous-time  OC  problem,  stated  by  equations 
(2.7)  -  (2.11)  introduced  in  section  2.2.1,  into  the  Hamiltonian  system  characterized 
by  (2.15).  Solving  the  ODEs,  analytically  if  possible,  but  most  often  numerically, 
associated  with  minimizing  H  in  (2.21)  -  (2.23),  applying  the  appropriate  boundary 
conditions,  and  applying  the  PMP  (2.25)  indirectly  minimizes  the  original  cost  func¬ 
tional  (2.7).  Direct  methods,  on  the  other  hand,  apply  an  appropriate  discretization 
to  the  OC  problem  directly  transforming  it  from  an  infinite-dimensional  continuous- 
time  problem  to  a  finite- dimensional  discrete-time  problem  which  then  can  be  solved 
using  nonlinear  programming  by  iteratively  searching  for  the  state  and  control  solu¬ 
tions  that  meet  optimality  conditions  while  satisfying  constraints  and  evaluating  the 
cost  function  to  verify  that  it  is  a  minimum  [71].  Direct  methods  are  attractive  since 
it  is  not  necessary  to  explicitly  derive  the  first-order  optimality  conditions  or  to  de¬ 
termine  if  the  inequality  constraints  are  active.  The  convergence  of  direct  methods  is 
less  sensitive  to  the  initial  guess  and  do  not  require  a  guess  for  the  costates  [7].  How¬ 
ever,  direct  methods  are  less  accurate  than  indirect  methods  and  there  may  actually 
be  several  minimizing  solutions  making  it  possible  for  the  result  to  be  incorrect  [78]. 
Since  direct  methods  do  not  provide  costate  information,  there  can  be  some  uncer¬ 
tainty  as  to  whether  the  NLP  optimal  solution  is  truly  the  optimal  solution  [50] .  This 
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subsection  briefly  summarizes  direct  methods  commonly  used  to  solve  OC  problems. 


2.3. 2.1  Nonlinear  Programming. 

The  NLP  problem  is  generally  stated,  as  presented  in  [61]  and  [71],  as: 


minimize  /( z) 

subject  to  h(z)  =  0  (2.45) 

g(z)  <  0 


where  z  £  Mn  is  the  vector  of  design  variables,  /( z)  is  the  objective  functional,  and 
h(z)  and  g(z)  are  the  constraints.  If  there  are  m  equality  constraints  and  p  inequality 
constraints,  then  h(z)  £  Mm  and  g(z)  £  W’.  The  first-order  necessary  conditions  for 
an  optimal  solution,  z*,  are  given  by  the  following  KKT  conditions: 


V/(z*)  +  ATVh(z*)+^TVg(z*) 

hi(  z*) 

9i{ Z1 


=  0 


=  0 

(*  =  !,- 

..,m) 

<  0 

■;P) 

(2.46) 

>  o 

■;P) 

=  0 

(*  =  !»■ 

■;P) 

where  V  indicates  the  gradient  of  the  argument  function  and  A  £  and  pL  £ 
are  Lagrange  multipliers  associated  with  the  equality  and  inequality  constraints 
respectively.  The  second-order  sufficient  conditions  for  optimality  are  obtained  from 
the  Hessian  (matrix  of  second-order  partial  derivatives)  of  the  Lagrangian  functional. 
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The  Lagrangian,  l,  is  given  by: 


Kz)  =  /(z)  +  ATh(z)  +  g(z)  (2.47) 

The  Hessian  of  the  Lagrangian,  evaluated  at  the  candidate  optimal  solution,  is  then: 

L(z*)  =  F(z*)  +  ATH(z*)  +  htG(z*)  (2.48) 

where  L(-),  F(-),  H(-),  and  G(-)  denote  the  Hessian  matrices  of  /(•),  /(•),  h(-),  and 
g(-)  respectively,  following  the  notation  in  [61].  The  second-order  necessary  condition 
is  that  in  addition  to  the  conditions  in  (2.46)  being  satisfied,  the  Hessian  of  the 
Lagrangian  function  (2.48)  be  positive-definite  at  the  candidate  z*. 

An  NLP  method  is  an  iterative  search  algorithm.  It  searches  through  the  space 
bounded  by  the  constraints  in  (2.45)  to  find  candidate  minimum  points,  z*,  that 
satisfy  the  constraints  with  the  lowest  cost.  The  optimality  conditions  in  equations 
(2.46)  and  (2.48)  are  checked  at  the  candidate  points  to  determine  whether  each  is 
a  minimum,  maximum,  or  saddle  point.  The  key  elements  in  the  search  algorithm 
are  determining  the  direction  in  the  space  and  the  size  of  the  step  that  should  be 
taken  in  that  direction  to  find  successive  candidate  points  [71].  Care  must  be  taken 
in  selecting  and  implementing  the  appropriate  search  algorithm  in  order  to  find  a 
minimum  solution  and  to  have  some  kind  of  guarantee  that  the  algorithm  is  globally 
convergent.  That  is  to  say  that  the  point  that  is  found  when  the  algorithm  terminates 
is  the  global  minimum  or  at  least  an  acceptable  local  minimum  in  a  small  vicinity  of 
the  global  solution  [61].  Rao  discusses  the  most  common  gradient  search  algorithms 
in  [71]  along  with  some  heuristic  ones  that  use  some  added  randomness  to  direct  the 
search.  Luenberger  and  Ye  present  several  algorithms  in  [61]  that  are  effective  in 
solving  nonlinear  constrained  optimization  problems.  They  also  present  proofs  that 
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specify  conditions  for  guaranteeing  local  and  global  minimum  solutions. 

Gradient  methods  are  effective  search  algorithms  using  the  gradient  vector  and 
Hessian  matrix  to  select  a  direction  of  descent  such  that  the  objective  function  /(x^+i) 
sufficiently  decreases  [61].  One  of  the  most  common  classes  of  descent  methods  uses 
the  following  to  select  subsequent  points  in  the  iterative  process: 


zk+i  =  zfc  -  akdk  (2.49) 

The  step  size,  ak,  is  calculated  to  minimize  /  and  the  search  direction,  d^,  is  given 
by: 

d/t  =  [F(zfc)]-1  V/(Zfc)T  (2.50) 

which  is  a  modified  Newton  method  [61].  The  steepest  descent  method  is  obtained  if 
the  inverse  of  the  Hessian  matrix,  [F(zfc)]-1,  is  replaced  with  the  identity  matrix  [61]. 
This  process  depends  on  the  second  partial-derivatives  of  /  existing  and  inversion  of 
the  resulting  Hessian  matrix  at  each  iteration.  Further  modifications  to  the  Newton 
method  exist  to  approximate  the  inverse  of  the  Hessian  to  avoid  costly  computation 
of  the  inverse  at  each  iteration  [61]. 

Shooting,  multiple  shooting,  and  collocation  methods  described  in  section  2.3.1 
can  be  applied  to  solve  the  direct  OC  problem  using  NLP  techniques.  The  concepts 
discussed  for  indirect  methods  are  similar  in  direct  methods,  with  some  modifications. 
Rao  presents  algorithms  for  shooting  and  multiple  shooting  in  [71]  and  Betts  discusses 
several  shooting  and  time-marching  methods  in  the  context  of  NLP  formulation  in 
[9].  Becerra’s  paper  [6]  discusses  translating  a  continuous-time  OC  problem  into 
an  NLP  problem  and  solves  an  example  problem  of  controlling  the  movement  of  a 
pendulum-like  system  described  by  ODEs  derived  using  Newton’s  Second  Law.  The 
main  idea  of  all  these  schemes,  as  stated  in  these  references,  is  that  they  provide 
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some  way  of  choosing  N  discrete  points  the  time  domain,  denoted  as  T;  (i  =  1, N), 
approximating  the  cost  function  by  replacing  the  integral  with  a  finite  summation, 
and  discretizing  the  dynamics  equations.  Conceptually,  the  discretization  process 
leads  to: 


J(x(t),u(t),t) 
X  =  f(x(t),u(t),t) 


j (x(Tj),  u(rj),  Tj) 
x(ri+i)  =  f (x(ri),  u(r*),  Ti) 


After  the  discretization  scheme  has  been  applied,  the  NLP  definition  (2.45)  looks  like: 


minimize  J(x(Tj),  u(rj),  rf) 

subject  to  x(ri+1)  -  f  (x(Tj),  u(t;),  rf)  =  0 


Umtn  Uj  ^  0,  Uj  U max  ^  0 

Xmm  Xj  A  0,  Xj  Xmaa;  A  0 

x(to)  =  x0 

x(^/)  =  x/ 

for  i  —  1, ...,  M 


(2.51) 


Other  constraints,  like  path  constraints,  can  be  added  to  (2.51)  as  dictated  by  the 
problem.  In  contrast  to  the  CV-based  indirect  methods,  bounds  on  the  states  can  be 
directly  included  in  the  NLP  formulation  as  inequality  constraints. 


2. 3. 2. 2  Direct  Shooting  and  Collocation. 

To  summarize  the  shooting  and  collocation  methods  using  the  NLP  formulation, 
it  can  be  stated  that  these  methods  are  simply  ways  of  representing  the  integral 
cost  function  as  a  discrete  sum  and  using  discrete  expressions  for  the  solution  of 
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the  dynamic  constraints.  Note  that  the  step  size  between  two  successive  points  is 
hi  =  ti+i  —  ti,  where  i  =  1  The  index  on  the  step  size  can  be  dropped  if 

the  step  sizes  are  equal.  A  quadrature  rule  for  evaluating  the  cost  functional  (2.7) 
is  necessary  and  should  be  selected  to  be  consistent  with  the  method  used  to  solve 
the  differential  equation  (2.8).  For  example,  if  an  R-K  scheme  is  used  to  solve  the 
differential  dynamics  equations,  then  an  R-K  scheme  of  the  same  order  should  be  used 
to  evaluate  the  cost  functional.  An  effective  way  to  accomplish  this  is  to  augment  the 
state  vector,  recalling  that  the  size  of  the  state  vector  is  n  x  1,  with: 


xn+i  =  g{x{t),u(t),t) 


(2.52) 


resulting  in  the  new  set  of  differential  equations: 


x(f) 


xn+i(t ) 


f(x(*),u(t),t) 


3(x(f),u(t),t) 


(2.53) 


The  cost  function  can  then  be  evaluated  after  the  time-marching  scheme  has  been 
applied  using: 

J  =  h(x(tf),t)  +xn+1(tf)  (2.54) 


In  both  the  shooting  and  multiple  shooting  methods,  the  control  is  approximated 
using  a  parameterization  of  the  form: 

N 

u(Q  ~  (2.55) 

i—  1 

The  coefficients,  a*,  are  determined  using  NLP  and  V'i(^)  are  basis  functions  of  choice. 
The  shooting  method  presented  in  [71]  and  [9]  can  be  summarized  as  an  NLP  problem 
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with  the  following  steps: 


1.  Make  initial  guess  of  unknown  coefficients,  aj. 

2.  Integrate  state  equation  (2.8)  over  the  interval  [to,tf]  using  the  parameterized 
control  by  applying  a  time-marching  method  discussed  in  section  2. 3.1.1. 

3.  Evaluate  the  error  between  the  solution  and  the  specified  boundary  conditions. 

4.  Evaluate  the  cost  functional  using  numerical  integration. 

5.  If  the  cost  is  not  at  a  minimum  value  and/or  the  boundary  conditions  have  not 
been  met,  then  update  the  values  of  a.;  using  a  gradient  method  and  repeat 
steps  2  through  5. 

Multiple  shooting  uses  the  same  procedure  but  applies  the  shooting  method  over 
each  of  the  IV  +  1  subintervals  in  [f0,  tf]  and  ensures  that  continuity  is  enforced  at 
the  interfaces  between  subintervals.  In  essence,  the  shooting  method  treats  the  OC 
problem  over  the  interval  as  a  single  problem,  whereas  multiple  shooting  treats  the 
OC  problem  as  IV  +  1  subproblems  connected  together  at  the  subinterval  interfaces. 

Direct  collocation  is  again  similar  to  collocation  methods  described  for  indirect 
methods.  The  difference  between  indirect  and  direct  collocation  is  that  the  differential 
equations  for  the  costates  do  not  need  to  be  derived  or  evaluated  and  a  quadrature 
scheme  is  used  to  approximate  the  integral  in  the  cost  functional  (2.7),  similar  to 
using  the  procedure  given  by  equations  (2.52)  -  (2.54).  Suitable  discrete  approxima¬ 
tions  of  the  states  are  made  using  some  type  of  polynomial  basis  functions,  which 
can  be  piecewise  or  global,  similar  to  equations  (2.39)  and  (2.42).  Additionally,  the 
direct  method  approximates  the  control  functions  in  the  same  way  that  the  states 
are  approximated.  These  equations  are  then  cast  in  the  NLP  formulation  in  (2.45) 
and  solved  for  the  states  and  controls  at  each  t*  that  satisfy  the  KKT  conditions,  are 
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within  the  boundaries  specified  by  inequality  constraints  on  the  states  and  controls, 
and  result  in  the  minimum  cost  value. 

At  this  point,  it’s  more  valuable  to  focus  on  direct  global  orthogonal  collocation 
since  it  is  a  key  piece  of  the  PSM  described  in  the  next  section.  Only  the  key  equa¬ 
tions  are  summarized  here  in  order  to  avoid  excessive  repetition  in  the  next  section 
where  further  discussion  will  be  essential  to  describing  the  PSM.  These  equations  are 
presented  in  a  general  sense  noting  that  there  will  be  variations  depending  on  the 
sets  of  collocation  points  and  orthogonal  polynomial  bases  chosen.  The  first  step  is 
to  select  the  set  of  N  collocation  points,  r,  (i  =  and  calculate  the  associated 

quadrature  weights,  wy,  using  possible  basis  sets  in  Table  2.2,  recalling  that  the  time 
domain  t  6  [to,tf]  needs  to  be  linearly  mapped  to  r  G  [—1,1].  Next,  a  Gaussian 
quadrature  rule  is  used  to  approximate  the  integral  in  the  cost  functional.  Re-writing 
(2.7)  to  replace  the  integral  with  Gaussian  quadrature  gives: 

f  _f  N 

J(x(t),u(t),t)  =  <f>(x(l ),tf)  +  f  2  °  ^^(x(rO,u(rO,rO  (2.56) 

i= 1 

The  state  and  control  functions  can  be  approximated  using  a  class  of  functions  known 
as  cardinal  functions1,  Ci,  given  by: 


N 

x(t)  ~  ^x(r*)C7(r) 

i=l 
N 

u(t)  &  ^u(Ti)Ci(r) 

1=1 

1The  term  cardinal  functions  was  first  used  by  Sir  Edmund  Whittaker  in  work  published  in  1915. 
It  refers  to  orthogonal  polynomial  or  trigonometric  basis  functions  used  to  approximate  functions 
using  polynomial  interpolation.  These  functions  have  the  property  that  the  ith  cardinal  function  is 
equal  to  one  at  the  ith  collocation  node  and  zero  elsewhere.  Cardinal  functions  are  also  known  as 
the  Lagrange  basis  and  the  Fundamental  polynomial  of  Lagrangian  interpolation  [13, 14] 


(2.57) 
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The  cardinal  functions  can  be  constructed  using  Chebyshev,  Legendre,  Laguerre,  or 
Hermite  orthogonal  polynomial  basis  functions  [14],  and  have  the  general  form: 


with  the  property: 


Ci(r) 


(pN+ijr) 

<i>N+l(Ti)(T  -  Ti) 


Ci{Tj)  =  Sij 


(2.58) 


(2.59) 


where  5lj  is  the  Kronecker  delta  function.  Lagrange  interpolating  polynomial  bases 
are  commonly  used  as  the  cardinal  functions  when  applying  direct  collocation  methods 
to  solve  OC  problems,  which  have  the  form: 

N 

Li(r ) = n 

3= 1 

jA* 

Using  (2.57)  with  Lagrange  interpolating  polynomial  bases  (2.60),  the  dynamics  in 
(2.8)  can  be  approximated  as: 

N 

x(t)  ~  Ex^)L,(r)  (2.61) 

1=1 

Again,  the  right-hand-sides  of  (2.8)  and  (2.61)  are  set  to  match  at  the  collocation 
points,  resulting  in  the  following  equality  constraint  for  the  NLP: 

N 

E  x(rj)Lj(r)  -  f(x(r),  u(r),  r)  =  0  (2.62) 

i= 1 

These  are  the  key  elements  necessary  to  transcribe  the  OC  problem  into  an  NLP 
problem  using  global  orthogonal  collocation.  The  NLP  can  be  constructed  to  solve 
for  the  states  and  controls  that  satisfy  first-  and  second-order  optimality  conditions, 
are  within  the  boundaries  specified  by  the  inequality  constraints,  and  result  in  the 


T  -  Ti 


Ti 


Ti 


(2.60) 
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minimum  cost  functional  value. 


2.3.3  Pseudospectral  Methods. 

The  discussion  of  numerical  techniques  for  solving  OC  problems  has  been  purpose¬ 
fully  presented  in  a  way  that  builds  up  to  the  PSM  presented  in  this  subsection.  The 
PSM  combines  many  of  the  previously  discussed  numerical  tools  into  a  very  powerful 
numerical  method,  and  has  been  generally  presented  as  global  orthogonal  collocation 
in  sections  2.3. 1.2  and  2. 3. 2. 2,  but  with  details  purposely  left  out  until  this  section. 
Indirect  methods  use  global  orthogonal  collocation  as  a  discrete  approximation  of 
the  HBVP  first-order  optimality  conditions  and  numerically  solve  for  x(t),  u(t),  and 
A (t)  that  minimize  'H(-),  thus  indirectly  minimizing  </(■),  using  root  finding  schemes. 
Direct  methods  use  global  orthogonal  collocation  to  approximate  x(t)  and  u(i),  and 
Gaussian  quadrature  to  approximate  the  integral  cost  functional,  J(x(£),  u(t),  t).  The 
discretized  continuous-time  OC  problem  can  then  be  solved  using  NLP  techniques. 
This  section  begins  with  a  brief  description  of  spectral  methods,  discusses  terminology 
to  distinguish  between  spectral  and  pseudospectral  methods,  followed  by  a  formulation 
of  the  direct  Gauss  Pseudospectral  Method  and  software  that  is  used  in  this  research. 

2. 3. 3.1  General  Discussion  of  Spectral  Methods. 

References  [14]  and  [33]  provides  excellent  development  and  application  of  spectral 
methods  to  solve  differential  equations.  Spectral  methods  are  fundamentally  a  family 
of  methods  used  to  solve  differential  equations  of  the  form: 

Lu(x )  =  f(x)  (2.63) 

where  L  is  a  general  linear  operator  that  can  represent  either  differential  or  integral 
operators.  To  solve  the  equation,  the  unknown  solution  is  represented  as  a  series  of 
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N  +  1  basis  functions  (j>n(x)  as: 


N 


u 


(2.64) 


and  substituted  into  (2.63)  to  give: 


Lun(x )  «  f(x) 


(2.65) 


The  goal  then  is  to  choose  the  expansion  coefficients,  an,  that  minimizes  the  residual 
function: 


Lun(x)  -  f{x)  =  R(x;  a0,  op, ...,  an) 


(2.66) 


The  beauty  of  spectral  methods  is  that  L  can  be  expressed  as  either  a  differentiation 
or  integration  matrix  translating  the  differential  equation  to  a  matrix  equation  where 
matrix  solvers  can  then  be  used  to  determine  the  expansion  coefficients.  With  non¬ 
linear  problems,  equation  (2.65)  generates  a  system  of  nonlinear  algebraic  equations 
that  necessitates  iterative  solution  techniques  such  as  Newton’s  method  or  a  variant 
thereof. 

The  difference  between  spectral  methods  lies  in  how  the  residual  function  is  min¬ 
imized.  The  term  spectral  methods  is  a  broad  term  to  include  two  main  categories  of 
methods:  interpolating  and  non-interpolating. 

Interpolating  methods  are  known  as  pseudospectral  methods.  These  methods  as¬ 
sociate  a  set  of  grid  points  with  a  selected  basis  function,  as  described  by  Table  2.2, 
and  find  the  expansion  coefficients  such  that  R(x]  a0,  ai, ...,  an)  =  0.  In  other  words, 
the  function  and  the  polynomial  representation  of  the  function  exactly  match  at  the 
grid  points.  Thus,  un(x)  converges  to  u{x)  as  N  increases  and,  because  the  points 
are  chosen  according  to  the  aforementioned  table,  the  convergence  rate  is  exponen- 
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tial,  also  called  spectral  convergence.  Furthermore,  the  error  of  the  pseudospectral 
approximation  is  ~  0[jj]N  [14].  Other  names  used  for  the  pseudospectral  method  are 
orthogonal  collocation ,  which  has  been  used  in  this  document,  and  method  of  selected 
points.  Pseudospectral  is  also  used  to  imply  that  global  basis  functions  are  being  used 
and  that  the  function  values  are  unknowns,  as  opposed  to  interpolation  where  the 
function  values  are  known  and  finding  the  expansion  coefficients  can  be  easily  solved 
for  in  order  to  approximate  the  function  with  an  interpolating  polynomial. 

The  keys  to  pseudospectral  methods  lie  in  selecting  appropriate  basis  functions  and 
collocation  points.  Boyd  presents  theorems  and  proofs  showing  convergence  of  several 
polynomial  bases  and  that  orthogonal  polynomial  basis  functions  with  collocation 
points  chosen  to  be  Gaussian  quadrature  points,  i.e.  the  roots  of  the  Nth,  (N  +  l)th, 
or  combination  of  the  two  as  given  in  Table  2.2,  provides  the  best  possible  result  for 
both  integral  and  derivative  forms  of  (2.63).  After  presenting  the  proofs,  Boyd  states: 

...since  quadrature  formulas  are  obtained  by  analytical  integration  of  the 
interpolating  polynomial,  it  follows  that  the  best  choice  of  quadrature 
points  is  also  the  best  choice  of  interpolation  points  and  vice-versa. 

and  that  as  a  rule  of  thumb: 

The  grid  points  should  be  the  abscissas  of  a  Gaussian  quadrature  associ¬ 
ated  with  the  basis  set. 

This  is  the  reason  why  collocation  methods  presented  earlier  are  chosen  to  solve  OC 
problems. 

Solving  the  pseudospectral  problem  involves  either  inverting  the  L  matrix  and 
performing  the  matrix  multiplication  L~x  f  or  using  a  Newton  solver  in  the  nonlinear 
case.  The  attractive  feature  of  this  method  is  that  only  evaluations  of  /  at  the 
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collocation  points  are  necessary  to  solve  the  problem.  In  contrast,  as  will  be  shown 
below,  the  Galerkin  spectral  method  requires  integration  of  the  function. 

Non-interpolating  methods  are  often  used  as  reference  to  Galerkin  methods.  The 
literature  can,  however,  be  confusing  on  terminology.  Some  references  use  the  term 
spectral  method  as  a  synonym  for  Galerkin  method ,  while  others,  like  [14],  use  the 
convention  adopted  in  this  document  where  the  term  spectral  methods  is  an  umbrella 
that  covers  both  pseudospectral  and  Galerkin  methods  as  well  as  some  others  not 
presented  here.  The  non-interpolating,  i.e.  Galerkin  method,  does  not  use  a  collo¬ 
cation  grid,  but  determines  the  expansion  coefficients  by  integration  resulting  from 
inner  products  with  test  functions.  The  general  formulation  and  solution  of  a  differ¬ 
ential  equation  using  the  Galerkin  method  is  briefly  shown  here.  Only  basic  details 
are  given  here  for  comparison  with  the  pseudospectral  method  and  as  motivation  for 
using  pseudospectral  rather  than  Galerkin  methods.  Refer  to  [14]  for  more  details  on 
the  Galerkin  method. 

Assume  that  L  in  equation  (2.63)  and  (2.65)  is  a  linear  operator,  either  differential 
or  integral.  The  Galerkin  method  solves  the  differential  equation  by  introducing  a  test 
function  or  weight  function  fa  and  using  it  to  take  the  inner  products  of  both  sides 
of  (2.63)  orthogonally  projecting  the  approximation  error  onto  the  space  spanned  by 
the  polynomial  bases  resulting  in  minimum  error.  Denoting  the  new  linear  operator 
H  and  function  g  after  the  inner  product  gives  the  following  matrix  equation: 


Ha  =  g 


(2.67) 


where  the  elements  of  H  and  g  are  given  by  the  inner  products: 


Hij  =  ( fa ,  Lfa)  for  i,j  =  1, ...,  N  +  1 


(2.68) 


9i  (0ij/) 
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The  inner  products  in  (2.68)  require  integration  operations  to  fill  each  element  in  H 
and  f,  which  makes  coding  the  Galerkin  method  more  complicated  than  the  pseu- 
dospectral  method.  Note  that  even  though  the  Galerkin  formulation  does  not  directly 
use  collocation  points,  they  do  become  necessary  to  apply  Gaussian  quadrature,  as 
described  in  section  2.3.2. 1,  to  numerically  approximate  the  inner  product  integrals. 
Therefore,  the  Galerkin  and  pseudospectral  methods  are  equivalent  if  the  Galerkin 
quadrature  points  are  the  same  as  the  pseudospectral  collocation  points  and  the  ac¬ 
curacy  of  the  pseudospectral  is  comparable  to  that  of  the  Galerkin  method.  Boyd 
states  that  a  Galerkin  method  using  N  expansion  terms  is  equal  in  accuracy  to  the 
pseudospectral  method  using  N  +  1  or  N  +  2  terms. 

Now  all  the  pieces  are  in  place  to  discuss  the  Gauss  Pseudospectral  Method  applied 
to  solve  OC  problems. 

2. 3. 3. 2  Gauss  Pseudospectral  Method  for  Optimal  Control. 

When  it  comes  to  solving  differential  equations,  either  Partial  Differential  Equa¬ 
tion  (PDE)s  or  ODEs,  there  are  many  PSMs  available.  The  differences  between  meth¬ 
ods  are  determined  by  the  orthogonal  polynomial  basis  functions  chosen  for  spectral 
representation  of  the  solution  and  the  collocation  points  used.  Reference  [14]  primar¬ 
ily  discusses  Fourier  and  Chebychev  methods,  [33]  presents  other  methods,  and  [60] 
presents  a  Chebyshev  method  for  solving  BVPs.  For  example,  Legendre  polynomial 
basis  functions  can  be  used  with  Legendre-Gauss  collocation  points,  Chebyshev  poly¬ 
nomials  with  Chebyshev  points,  and  so  on.  Lagrange  interpolating  polynomial  basis 
functions  can  always  be  used  in  place  of  the  other  basis  sets  [14].  The  HBVP  can 
be  solved  using  any  of  these  combinations  of  basis  functions  and  collocation  points, 
such  as  in  [27]  where  Lagrange  polynomial  bases  combined  with  Chebyshev  colloca¬ 
tion  points  were  used.  However,  since  the  first-order  optimality  conditions  of  the  OC 
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problem  given  by  (2.21)  -  (2.23),  the  associated  boundary  conditions,  and  switching 
conditions  of  the  inequality  constraints  (2.10)  may  be  extremely  difficult  to  derive, 
it’s  attractive  to  use  the  direct  PSM  to  solve  OC  problems,  however  the  additional  re¬ 
quirement  of  minimizing  the  cost  functional  makes  direct  OC  solution  methods  more 
difficult  than  simply  solving  a  set  of  differential  equations  [50]. 

The  most  common  PSMs  used  to  directly  solve  nonlinear  OC  problems  have  the 
same  general  procedure  that  parameterizes  the  states,  x(i)  G  Mn,  and  controls, 
u(t)  G  Mm,  using  Lagrange  interpolating  polynomials,  discretizes  the  constraints 
(2.8)  -  (2.10)  and  the  general  Bolza  cost  functional  (2.7)  using  a  quadrature  rule, 
converting  the  differential  dynamic  constraints  to  algebraic  constraints,  and  applies 
an  NLP  solver  to  find  the  solution  to  the  resulting  parameter  optimization  prob¬ 
lem  [7,8,28,29,50].  Some  of  the  most  common  direct  PSMs  are  the  Legendre  Pseu- 
dospectral  Method  (LPM)  [26,28],  Chebyshev  Pseudospectral  Method  (CPM)  [29], 
GPM,  and  Radau  Pseudospectral  Method  (RPM)  [7,36,50].  Several  software  pack¬ 
ages,  a  sampling  of  which  are  listed  in  [50],  have  been  developed  to  directly  solve 
the  continuous-time  OC  problem  by  discretizing  and  transcribing  it  into  an  NLP 
problem  using  this  general  procedure.  Two  common  packages,  both  written  for  use 
in  MATLAB®and  using  SNOPT  [42]  as  the  NLP  solver,  are  DIDO  [1],  which  imple¬ 
ments  the  Legendre  pseudospectral  method  developed  in  [28],  and  GPOPS  [72],  which 
implements  the  GPM  developed  in  [8],  [50],  and  [7]. 

Selecting  which  PSM  to  use  to  solve  the  direct  formulation  of  the  OC  problem  can 
be  confusing,  ft’s  known  that  solving  the  HBVP  provides  guarantees  on  meeting  the 
optimality  conditions  [8,29].  Significant  research  efforts  have  focused  on  determining 
which  method  to  use  for  specific  problem  sets,  for  example  finite  horizon,  infinite 
horizon,  etc.  [30,35].  Regardless  of  the  method  chosen,  it  is  advantageous  to  select  a 
direct  PSM  that  has  been  shown  to  have  some  equivalence  to  the  numerical  solution  of 
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the  continuous  CV-based  HBVP.  One  such  method  is  the  GPM,  which  Benson  derived 
in  [7].  Benson  showed  how  the  continuous-time  OC  problem  can  be  transcribed 
to  a  NLP  problem  using  Lagrange  interpolating  polynomial  bases  of  degree  N  to 
parameterize  the  states  and  controls  at  N  +  1  LG  collocation  points.  Benson  also 
proposed  and  provided  proof  of  the  following  theorem: 

The  Karush-Kuhn-Tucker  (KKT)  conditions  of  the  NLP  are  exactly  equiv¬ 
alent  to  the  discretized  form  of  the  continuous  first-order  necessary  condi¬ 
tions  of  the  Bolza  problem.  Furthermore,  a  costate  estimate  can  be  found 
from  the  KKT  multipliers,  A*,  [costates  of  the  pseudospectral  represen¬ 
tation  of  the  OC  problem],  and  Lagrange  multipliers,  iz,  that  satisfy  the 
pseudospectral  approximation  to  the  costate  dynamics. 

Benson  was  able  to  conclude  that  solving  the  OC  problem  using  direct  transcription 
to  an  NLP  problem  is  “exactly  equivalent  to  solving  the  discretized  form  of  the  contin¬ 
uous  first-order  necessary  conditions”  given  by  the  HBVP.  Reference  [8]  summarizes 
the  GPM  development,  pseudospectral  representation  of  the  CV-derived  HBVP,  and 
proves  equivalence  of  the  KKT  conditions  of  the  NLP  and  the  first-order  continuous 
necessary  conditions  of  the  HBVP. 

Huntington  extended  Benson’s  work  in  [50]  by  adding  inequality  path  constraints 
to  the  GPM  formulation  and  again  showed  equivalence  between  the  KKT  multipliers 
and  the  first-order  optimality  conditions  of  the  HBVP.  He  also  investigated  the  same 
equivalence  of  the  LPM,  based  on  LGL  collocation  points,  and  the  RPM,  based 
on  LGR  collocation  points,  and  showed  mapping  relationships  between  the  KKT 
multipliers  and  the  costates  of  the  HBVP,  noting  that  the  GPM  had  the  most  direct 
mapping  relationships,  particularly  at  the  endpoints.  Fahroo  and  Ross  have  also 
studied  the  equivalence  of  the  LPM  KKT  optimality  conditions  and  the  optimality 
conditions  of  the  HBVP,  noting  that  the  KKT  multipliers  “...satisfy  a  discrete  analog 
of  the  costate  dynamics”  at  the  interior  points  giving  a  check  that  the  solution  meets 


53 


optimality  conditions.  Similar  to  Benson,  Huntington  concluded  that,  although  all 
three  methods  provide  a  mapping  between  the  NLP  KKTs  and  the  HBVP  costates, 
“...solving  the  NLP  derived  from  Gauss  pseudospectral  transcription  is  equivalent 
to  applying  Gauss  pseudospectral  discretization  to  the  continuous-time  first-order 
optimality  conditions.”  The  main  take-away  from  Huntington’s  work  is  that  while 
researchers  are  currently  investigating  convergence  of  the  direct  transcription  to  the 
HBVP,  equivalence  of  the  KKT  multipliers  with  first-order  optimality  conditions  gives 
confidence  that  an  optimal  solution  has  been  found. 

The  key  equations  transforming  the  continuous  OC  problem  to  a  parameter  op¬ 
timization  problem  used  by  GPOPS  and  solved  with  SNOPT  as  derived  in  [7,50] 
and  summarized  in  [8,  72]  are  paraphrased  below  since  GPOPS  is  a  key  tool  used 
extensively  in  the  research  presented  in  this  dissertation.  Some  repetition  of  previous 
sections  will  be  evident,  but  is  necessary  for  continuity  in  describing  how  GPOPS 
discretizes  the  continuous  OC  problem  and  transcribes  it  to  the  NLP  problem  for 
SNOPT  to  solve.  In  discussing  the  GPOPS  implementation,  there  will  be  some  nota¬ 
tion  deviation  from  the  general  discussion  presented  in  previous  sections.  Again,  this 
is  necessary  for  the  sake  of  continuity  in  discussing  the  specific  implementation. 

The  continuous  Bolza  OC  problem  can  be  stated  as  [8,72]:  Determine  the  state, 
x(r)  G  M”,  and  control,  u(r)  G  Rm,  and  the  terminal  time,  tf  if  unknown,  to  minimize 
the  cost  functional: 

J  =  $(x(— 1),  t0,  x(l),  tf)  +  tf  2  t0  J  9(x(T),u{T),T]t0,tf)dT  (2.69) 

subject  to  the  differential  (dynamic),  boundary,  and  path  constraints,  respectively: 

=  ~  2  f°f(x(vb  u(t),  r;  t0,  tf)  (2.70) 
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0(x(-i),  t0,  x(i),  tf)  =  0  el" 


(2.71) 


C(x(r),  u(r),  t;  t0,  tf)  <  0  G  Mc  (2.72) 

Note  that  equations  (2.69)  -  (2.72)  have  been  written  in  the  time  variable  r  to  be 
in  the  proper  domain  for  the  GPM.  Linear  transformations  between  t  G  [to,tf\  and 
r  G  [— 1, 1]  are  accomplished  by: 


t  = 


tf  ~  to 
2 


r  + 


tf  +  to 
2 


(2.73) 


The  GPM  process  is  to  discretize  the  continuous  equations  (2.69)  -  (2.72)  and  tran¬ 
scribe  them  into  an  NLP  problem.  The  process  begins  by  choosing  N  LG  collocation 
points  G  (—1, 1),  noting  that  the  endpoints  are  not  included  in  the  set.  The  states, 
x(t),  and  controls,  u(r),  are  approximated  using  N  or  N  +  1  Lagrange  interpolating 
polynomials  of  the  form: 


N 


x(r)«X(r)  =  ^X(r.)Lf(r) 

i=0 

(2.74) 

N 

u(t)«U(t)  =  ^U(n)L“(T) 

(2.76) 

i= 1 


where  lower  case  symbols,  x(r)  and  u(r),  denote  the  continuous  state  and  control 
vectors  while  the  capitalized  symbols,  X(r)  and  U(r),  represent  the  corresponding 
polynomial  approximations.  Additionally,  superscripts  x  and  u  are  used  to  distinguish 
between  Lagrange  polynomials  representing  the  state  and  control  approximations, 
respectively.  The  Lagrange  polynomials  have  the  form: 

N 

UM  =  II  L^J1  (i  =  0  •~’N)  (2.76) 

75  “  T> 
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}  =  1,-,N) 


(2.77) 


N 

•  i  ' i  '  1 

J=i  j 

and  satisfy  the  isolation  property: 

{1  if  i—j 

(i  —  0, N) 

0  if  i  ^  j 

{1  if  i  =  j 

(i  =  1, N) 

0  if  i  7^  j 

The  differential  dynamic  constraints  (2.70)  are  approximated  by  differentiating  (2.74): 


N 

i(r)  «  X(r)  =  s^jx{ri)Li{r) 

i= 0 


(2.78) 


The  derivatives  of  the  Lagrange  polynomials  evaluated  at  the  LG  collocation  points 
can  be  written  in  a  differentiation  matrix,  D,  which  has  dimension  N  x  N  +  1,  and 
whose  elements  are  expressed  as: 


N  n  i =0  (Tk  Tj) 
Dki  —  Li{rk)  =  ^  — 

t“o  IL=o in-Tj) 


(2.79) 


where  the  counting  indices  are  k  =  1  and  i  =  0 The  dynamic  constraints 

(2.70)  are  discretized  and  transformed  into  algebraic  constraints  suitable  for  the  NLP 
solver  using  (2.74),  (2.75),  (2.78),  and  (2.79)  as  follows: 


-DfcjXj — ,f  —  °f(Xfc,  Ufc,  Tfcj  t0,  tf)  —  0  (k  —  1, ...,  N)  (2.80) 

i= 0 

For  clarity,  shorthand  notation  definitions  used  in  the  GPM  development  are  sum- 
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marized  in  the  following  list: 


•  Xk  =  X(rfc)  G  Mn  for  k  —  1, N. 

•  Ufc  =  U(rfc)  G  Mm  for  k  =  1, N. 

•  Xo  =  X(— 1),  where  r  =  —1  is  not  a  collocation  point. 

•  X/  =  X(l),  where  r  =  1  is  not  a  collocation  point. 

•  9k  =  Ufc,  Tfc;  t0,  t/) 

•  ffc  =  f  (X/-,  Ufc,  Tfc,  to,  ^/) 

•  Cj'fe  =  Cj'fXj;,  Ufc,  Tfc,  to;  tj) 

•  Hk  =  ^(Xfc,  Afc,  Af,  fik,  Ufc,  Tk]t0,tf),  where  the  Hamiltonian,  and  La¬ 
grange  multipliers,  A*,,  A^p,  and  jlk  are  dehned  later. 

Since  X(r)  and  U(r)  are  not  collocated  at  the  endpoints,  it’s  necessary  to  approximate 
the  terminal  state,  X^,  using  Gaussian  quadrature: 

t  -t  N 

X/  =  X0  +  wkf(X.k,  Ufc,  rk ;  t0,  t/)  (2.81) 

fc=i 

The  NLP  cost  function  will  be  a  discretized  form  of  (2.69),  which  is  again  generated 
using  Gaussian  quadrature  to  approximate  the  integral  term.  The  discretized  cost 
functional  is: 

t  -t  N 

J  =  <E>(X0,  to?  X/,  tf)  +  wkg(X. k,  Ufc,  rk,  t0,  tf)  (2.82) 

k=  1 

The  last  necessary  items  for  the  NLP  transcription  are  discretized  forms  of  the  bound¬ 
ary  constraints  (2.71)  and  path  constraints  (2.72),  stated  as: 

4>{Xi),to,Xf,tf)  =  0  (2.83) 
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and: 


C(Xfc,  Ufc,  Tk]  t0,  tf)  <  0  (k  —  l,...,N)  (2.84) 

With  the  discretization  given  above,  the  NLP  problem  can  be  stated  as:  find  X& 
and  Ufc  that  minimizes  (2.82)  subject  to  (2.80),  (2.81),  (2.83),  (2.84),  and  X(to)  =  X0. 
GPOPS  calls  the  SNOPT  NLP  solver  to  find  the  solution,  which  is  an  approximate 
solution  to  the  continuous  Bolza  OC  problem. 

The  NLP  solver  searches  for  the  solution  that  satisfies  the  KKT  conditions.  The 
KKT  conditions,  similar  to  those  introduced  in  section  2. 3. 2.1,  are  listed  below,  again 
paraphrasing  [8].  They  are  found  using  the  augmented  cost  functional,  also  known  as 
the  Lagrangian,  which  is  analogous  to  (2.13),  constructed  by  adjoining  the  discrete 
constraints  with  the  discretized  cost  functional  though  introduction  of  Lagrange  mul¬ 
tipliers  is,  /2,  and  A.  The  augmented  cost  function  is  given  by: 


Ja=  $ 


EN  ~T  i  sr^N  ~  T 

k=1wkg-v  c/)-^k=1vkC 


—  \^N  \T  \x^N  n  x 

l^k=l  2^i=0  Uki^i  2 


(2.85) 


-A; 


X/  -  x0  -  ^  eJLi  wkt 


The  arguments  of  the  functions  have  been  dropped  for  readability.  Analogous  to  the 
development  of  the  first-order  necessary  conditions  of  the  continuous  OC  problem 
developed  in  section  2.2.2,  the  KKT  conditions  are  found  by  setting  to  zero  the 
partial  derivatives  of  Ja  with  respect  to  X0,  Xfc,  Xj,  U*,,  Ak,  A f,  fik,  u,  t0,  and  tf. 
Defining  the  discretized  Hamiltonian,  T-Lk,  as: 

Uk  =  gk  +  ( ^  +  A^)  fk  -  -?—^Ck  (2.86) 

\wk  /  tf-tQwk 


and  substituting  into  the  equations  for  the  partial  derivatives,  yields  the  following  set 
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of  KKT  conditions  that  must  be  satisfied  for  an  optimal  solution: 


N 


tf  ~  t0 


ffc  =  0 


(2.87) 


i= 0 


N 


A, 


El  1\-A  ~  T  \  Wo  —  ~  1  V ^  Wo 

I  — - — b  Af  —~Dik  +  Af  —Dik 


N 

T  \  -  Wi 


.  .  .  W. 

1=1 

tf  -  t0 


Wk 


i= 1 


'<9Xz; 


Wk 


Wk 


Oh. 


(2.88) 


_2 _ <9Cfc 

dX-k  tf  to  <9Xfc 


+ 


=  0 


<%•  ,  f  +  at  i  i?*.  _  2  Al  ^cfc 

3Ufc  <9Ufc  t/-t0u;fedUfc 

0(XO,  to,  X/,  t/)  =  0 


=  0 


~t  <9<f>  d<±> 

A°  +  Wr  ~  V  ~  =  0 


A, 


>-0 


a# 


<9X, 


dX. 


o 


oTS*-=o 


f 


dX 


f 


tf  —  to  \  ^  dHk  1  \  ^  9$  Td(j) 

wr° 


2 

tf  - 10 


k= 1 
N 


N 


dHk  1  \  ^  ^ 

2^Wk~nT7  +  9  2^Wknk  + 


k= 1 


k= 1 

cfc  <  0 


dtf 


~rd<t> 

^  oT  =  0 
dtf 


(2.89) 

(2.90) 

(2.91) 

(2.92) 

(2.93) 

(2.94) 

(2.95) 


i-ijk  ■ 


=  0  when  Cjk  <  0 
<  0  when  Cjk  =  0 


(2.96) 


X,  =  Xn 


tf  ~  to 


N 


^  ^  Wki (Xfc,  Ufc,  Tfc,  to,  tj) 


fc=l 


(2.97) 
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0 


dgk  _  /  Afc  £r\  dik_  2  yTk  dCk 
dXk  \  wk  FJdXk  tf-to  wk  dXk 

(2.98) 

The  pseudospectral  discretization  and  NLP  transcription  formulation  given  by 
equations  (2.74)  -  (2.84)  form  the  basis  for  GPOPS  software.  The  NLP  problem  is 
then  solved  by  calling  SNOPT  to  find  the  solution  that  minimizes  the  cost  functional 
(2.82)  and  satisfies  the  KKT  conditions  (2.87)  -  (2.98).  Reference  [8]  continues  by 
formulating  the  pseudospectral  numerical  solution  to  the  HBVP  to  show  equivalence 
between  the  KKT  multipliers  and  costates  of  the  numerical  HBVP  approximation, 
showing  that  the  GPM  generates  a  solution  that  well  approximates  the  numerical  so¬ 
lution  to  the  HBVP.  Therefore,  the  GPM  as  coded  in  GPOPS  is  a  powerful  numerical 
tool  for  solving  many  types  of  OC  problems. 

2.3.4  Numerical  Methods  for  Optimal  Control  Summary. 

Topics  related  to  numerically  solving  both  indirect  and  direct  formulations  of  the 
OC  problem  have  been  described  in  this  section.  Numerical  methods  applied  to  the 
indirect  formulation  solve  the  OC  problem  that  has  been  converted  to  the  Hamilto¬ 
nian  Boundary  Value  Problem  while  methods  applied  to  the  direct  formulation  solve 
the  OC  problem  directly  by  transcribing  the  original  problem  into  a  parameter  opti¬ 
mization  problem  and  applying  nonlinear  programming  techniques.  This  section  has 
been  presented  purposely  to  discuss  basic  numerical  techniques  that  when  combined 
result  in  very  powerful  pseudospectral  methods,  particularly  the  Gauss  Pseudospec¬ 
tral  Method.  References  have  also  been  provided  that  show  equivalence  between  the 
direct  pseudospectral  and  indirect  pseudospectral  solutions,  adding  confidence  that 
direct  pseudospectral  transcription  methods  generate  solutions  that  meet  first-order 
optimality  conditions  derived  via  the  Calculus  of  Variations.  Lastly,  a  brief  summary 
of  the  key  equations  in  the  development  of  the  Gauss  Pseudospectral  Method  and  the 


k= 1 
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Karush-Kuhn- Tucker  optimality  conditions  were  presented  in  order  to  put  together 
the  key  concepts  of  collocation  and  nonlinear  programming  applied  to  solving  OC 
problems.  This  formulation  is  the  basis  for  the  Gauss  Pseudospectral  Optimization 
Software  which  performs  pseudospectral  discretization  of  the  continuous  OC  problem, 
formulates  the  nonlinear  programming  problem,  and  calls  the  nonlinear  programming 
solver  to  find  the  solution  that  meets  optimality  conditions.  This  is  a  powerful  soft¬ 
ware  tool  that  will  be  combined  with  stochastic  methods,  which  will  be  discussed  in 
the  next  section,  to  solve  OC  problems  for  systems  with  uncertainties. 

2.4  Numerical  Methods  for  Stochastic  Differential  Equations 

The  goal  of  the  research  presented  in  this  dissertation  is  to  numerically  solve 
stochastic  OC  problems,  or  equivalently  trajectory  optimization  problems,  with  un¬ 
certainties  on  the  states,  constraints,  or  both.  The  material  presented  thus  far  applies 
to  deterministic  problems.  That  is  to  say  that  the  state  of  the  system  is  perfectly  mod¬ 
eled  by  differential  dynamic  equations  or  measurements  of  the  system  are  certain.  In 
reality,  there  are  a  multitude  of  uncertainties  to  cope  with  in  trajectory  optimization 
problems  such  as  noisy  measurements,  uncertainties  in  system  models,  uncertainties 
in  obstacle  locations,  random  disturbances  such  as  wind  gusts,  and  so  on.  The  pres¬ 
ence  of  these  uncertainties  makes  the  problem  more  challenging,  but  does  not  make 
the  methods  described  for  solving  OC  problems  invalid.  Indeed,  the  aforementioned 
methods  can  be  combined  with  additional  methods  to  address  randomness.  In  this 
section,  brief  descriptions  of  concepts  related  to  the  solution  of  stochastic  problems 
are  presented.  The  discussion  will  lead  to  Generalized  Polynomial  Chaos  methods 
that  when  combined  with  deterministic  methods  provides  a  powerful  tool  for  solving 
stochastic  OC  problems. 
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2.4.1  Survey  of  Stochastic  Methods. 


Solving  nonlinear  OC  problems  is  very  difficult  and  the  literature  is  especially 
sparse  on  the  topic  of  nonlinear  stochastic  OC  [22],  Many  nonlinear  OC  techniques 
addressing  problems  where  systems  are  subject  to  uncertainties  (noise),  i.e.  stochas¬ 
tic  OC  problems,  attempt  to  linearize  the  systems  and  make  simplifying  assumptions 
about  the  noise.  Several  well- developed  tools  become  available  by  considering  linear 
systems  with  additive  white  (Gaussian)  noise.  Linear  filtering  techniques  enable  sep¬ 
aration  of  noise  from  the  signal  of  interest,  for  example  measurement  of  system  states 
or  rejection  of  noise  in  the  plant  model,  by  minimizing  the  mean-square  estimation 
error.  Wiener  Liters  (frequency  domain)  [15],  and  Kalman  Liters  [51,52]  (state-space 
domain)  provide  such  estimates.  "Hoo  estimation  provides  another  method  for  esti¬ 
mating  system  states  by  minimizing  the  inhnity-norm  of  the  gain  between  disturbance 
inputs  and  estimation  error  [20].  The  resulting  state  estimates  can  then  be  used  as 
inputs  to  LQR  controllers.  TO  control  is  similar  to  the  LQR,  but  whereas  the  LQR  is 
not  a  stochastic  control  algorithm,  the  T-L-2  controller  rejects  noise  and  minimizes  the 
closed-loop  system  2-norm.  Linear  Lltering  and  estimation  techniques  work  well  for  a 
wide  range  of  engineering  problems,  but  Lltering  applied  to  nonlinear  systems  is  much 
more  difficult  since  state  estimation  and  control  can  not  be  separated  from  each  other, 
noise  rejection,  state  estimation,  and  control  objectives  compete  with  each  other,  and 
a  given  input  probability  distribution,  Gaussian  for  example,  generally  does  not  map 
to  the  same  distribution  in  the  output  [31,46].  That  said,  there  are  effective  non¬ 
linear  Lltering  techniques  available,  such  as  particle  Liters,  that  work  quite  well  but 
may  be  difficult  to  implement.  A  survey  of  nonlinear  Lltering  techniques  is  presented 
in  [18].  For  these  reasons,  other  approaches  are  needed  to  solve  the  general  nonlinear 
stochastic  OC  problem. 

Representative  nonlinear  stochastic  control  methods  based  on  DP,  which  are  fun- 
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damentally  sequential  decision-making  algorithms,  are  presented  in  [56],  [22]  and  [73]. 
Reference  [56]  discretizes  the  continuous  state  dynamics  using  Markov  Chains,  which 
are  random  process  where  the  state  of  the  system  at  time  step  k  +  1  only  depends 
on  the  state  of  the  system  at  step  k.  The  key  element  in  the  Markov  structure  is 
definition  of  a  transition  model,  which  is  stated  as:  given  the  system  state  xk  =  s  at 
time  step  k,  the  probability  that  the  system  will  reach  state  Xk+\  =  s'  when  control 
uk  is  applied  is  P(s,u,s')  [74],  Written  in  another  way,  the  transition  probability  is: 

Ps,s'(uk )  =  Prob{xk+ 1  =  s' \xk  =  s,uk}  (2.99) 

Kushner  states  that  minimizing  a  DP  cost  functional  using  the  Markov  chain  repre¬ 
sentation  of  the  stochastic  processes  results  in  optimal  solutions  that  are  good  approx¬ 
imations  to  the  original  problem.  Rogers  presents  a  slightly  different  DP  approach, 
again  using  Markov  chains,  where  “pathwise”  deterministic  optimization  is  conducted 
on  many  randomly  sampled  trajectories  to  find  an  approximate  solution.  He  notes 
that  this  is  a  novel  approach  in  solving  stochastic  OC  problems,  but  numerical  im¬ 
plementation  needs  further  study  [73].  The  difficulties  of  applying  these  DP-based 
methods  are  that  it  is  not  clear  how  to  define  the  transition  probabilities  based  on 
the  randomness  in  the  OC  problem  and  algorithms  for  actually  implementing  these 
methods  are  not  clearly  stated. 

Reference  [22]  uses  a  “backward  search  algorithm”  that  is  based  on  Bellman’s  prin¬ 
ciple  of  optimality  to  generate  global  control  solutions  to  stochastic  optimal  control 
problems  with  fixed-state  terminal  conditions  and  state  and/or  control  constraints 
that  “can  handle  strongly  nonlinear  and  non-smooth  systems,  can  include  various 
state  and  control  constraints  and  leads  to  global  solutions.”  Crespo  and  Sun’s  method 
can  be  summarized  as: 

•  Formulate  the  SDE  using  random  inputs  modeled  as  independent  Wiener  pro- 
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cesses,  which  are  also  known  as  Brownian  motion ,  and  is  a  method  of  modeling 
uncertainty  as  a  differential  equation  by  passing  white  Gaussian  noise  through 
an  integrator  [15,63]. 

•  Specify  the  cost  as  the  expectation  of  a  Bolza-type  functional. 

•  Apply  Bellman’s  principle  of  optimality. 

•  Implement  a  backward  search  algorithm  beginning  at  the  terminal  boundary 
conditions  and  searching  backward  through  the  space  and  time  domains  to  find 
the  optimal  solution. 

Through  three  examples,  Crespo  concluded  that  the  “method  leads  to  controls  with 
excellent  performance”  and  indicated  that  improvements  and  further  studies  are 
needed  to  solve  higher-dimensional  problems  and  “rigorous  study  of  convergence  and 
stability  of  the  method  has  yet  to  be  done”  [22], 

Girardeau  compared  two  sample-based  stochastic  OC  schemes  in  [43]:  a  scenario 
tree  method  and  particle  method.  He  considered  a  continuous-time  finite  horizon 
problem  of  controlling  a  dynamical  system  perturbed  by  exogenous  noise,  assumed 
to  be  uniformly  distributed,  using  a  mean  squared  error  (MSE)  cost  function.  Both 
methods  involve  discretization  of  the  state  space,  time  domain,  constraints,  and  cost 
function  and  essentially  become  large  sets  of  deterministic  problems.  In  the  scenario 
tree  method,  he  defined  a  scenario  as  a  set  of  sampled  noise  values  at  each  time 
step  in  the  discrete  time  domain.  The  method  basically  solves  the  discrete  problem 
using  a  DP  approach  that  uses  knowledge  of  all  previous  state,  control,  and  noise 
values  to  sequentially  determine  the  state  feedback  control  function  at  the  current 
state  and  sampled  noise  value  that  minimizes  the  expected  cost  at  future  points  in 
time.  The  control  solution  is  then  used  to  propagate  the  state  to  the  next  time  point 
through  the  discretized  dynamics.  A  numerical  example  indicates  that  the  method 
is  computationally  expensive  since  numerical  integration  of  the  MSE  cost  function 
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is  required  at  each  time  step  and  that  as  the  time  horizon  increases,  the  number 
of  scenarios  needed  to  achieve  an  accurate  solution  exponentially  increases.  The 
particle  method  proposed  is  a  mix  of  stochastic  and  dynamic  programming  where 
the  necessary  optimality  conditions  are  formulated  using  variational  techniques,  i.e. 
a  costate  vector  is  generated,  and  then  solved  using  sampling.  The  solution  is  found 
using  gradient  search  methods  where  the  sample  grid  adaptively  discretizes  portions  of 
the  state  space  that  are  most  visited  by  the  resulting  deterministic  optimal  solutions. 
Girardeau  showed  that  this  method  does  not  have  the  same  exponential  growth  of 
scenarios  as  in  the  scenario  tree  method,  but  becomes  unsolvable  as  the  dimension  of 
the  state  space  increases.  While  these  methods  seem  to  be  able  to  solve  a  nonlinear 
stochastic  OC  problem,  the  formulations  and  examples  do  not  include  fixed  final 
states,  thus  it’s  not  evident  how  they  could  be  applied  or  adapted  to  solve  a  broad 
range  of  OC  problems. 

It  was  noted  in  section  2. 3. 3. 2  that  solving  deterministic  OC  problems  using  the 
GPM  generates  solutions  that  well  approximate  the  numerical  solution  to  the  HBVP. 
If  OC  problems  are  thought  of  as  of  a  system  of  ODEs,  as  in  the  indirect  HBVP  for¬ 
mulation,  then  tools  for  solving  SDEs  may  be  available  which  can  be  combined  with 
the  GPM  to  solve  stochastic  OC  problems.  As  will  be  shown  later,  solving  SDEs  will 
ultimately  depend  on  repeated  application  of  deterministic  ODE  solvers  and,  to  fore¬ 
shadow,  there  may  be  spectral,  or  more  specifically  pseudospectral,  representations 
of  stochastic  solutions  that  are  accurate,  computationally  efficient,  and  easy  to  use. 
A  survey  of  solution  methods  for  SDEs,  specifically  Monte- Carlo  Simulation ,  Pertur¬ 
bation  Methods ,  Moment  Equations ,  Operator  Methods,  and  Generalized  Polynomial 
Chaos ,  are  summarized  in  [87]  and  [88]  and  presented  with  greater  detail  in  [39].  The 
discussion  below  is  a  paraphrased  synthesis  of  the  information  in  these  reference  with 
complementary  references  inserted  as  appropriate. 
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The  most  common  technique  for  solving  SDEs  is  the  brute  force  Monte-Carlo  Sim¬ 
ulation  (MCS)  method.  The  MCS  method  generates  random  samples  of  a  stochastic 
variable  taken  from  an  assumed  probability  distribution  function  (PDF).  Each  ran¬ 
dom  sample  is  subsequently  inserted  into  the  SDE,  making  it  a  deterministic  problem 
that  can  be  solved  using  deterministic  differential  equation  solution  methods,  such 
as  those  discussed  in  section  2.3,  or  existing  ODE  and  PDE  numerical  solvers  in 
Matlab®.  A  set,  or  ensemble,  of  solutions  is  collected  from  which  the  statistical 
information,  such  as  expected  value  (mean),  variance,  and  covariance,  is  calculated. 
Convergence  is  checked  at  some  arbitrary  interval,  after  every  500  or  1000  samples 
for  example,  and  is  said  to  have  been  achieved  when  the  change  in  expected  value  be¬ 
tween  successive  intervals  is  below  a  desired  tolerance.  The  solution  to  the  problem  is 
the  combination  of  expected  value,  the  most  likely  solution  to  the  stochastic  problem, 
and  second  moment  variance  and  covariance  information,  which  characterizes  the  dis¬ 
tribution  of  the  solution  by  showing  how  much  the  solution  varies  from  the  mean  as  a 
function  of  uncertain  parameters  and  correlations  between  solution  variables,  respec¬ 
tively.  The  MCS  method  is  relatively  easy  to  implement  using  existing  deterministic 
differential  equation  solvers,  will  eventually  converge  to  a  solution  regardless  of  the 
PDF  of  the  process,  and  is  not  limited  by  the  amount  of  uncertainty  in  the  variable. 
However,  it’s  well  known  that  the  mean  converges  slowly,  on  the  order  of  -^=,  where 
k  is  the  number  of  realizations  (samples)  of  the  random  variable,  which  implies  high 
computational  burden,  especially  in  problems  with  multiple  random  variables.  Even 
though  the  MCS  is  computationally  burdensome,  it  is  often  capable  of  quantifying 
effects  of  uncertainty  in  problems  were  other  methods  seem  to  fail  and  is  a  valuable 
method  in  verifying  results  obtained  using  other  methods  [39]. 

Perturbation  Methods ,  Moment  Equations ,  and  Operator  Methods  offer  alterna¬ 
tives  to  solving  SDEs,  but  will  not  be  used  in  this  research  and  are  only  mentioned 
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here  for  the  sake  of  completeness.  Perturbation  Methods  expand  the  field  of  the 
random  variables  about  their  mean  values  using  truncated  Taylor  series  expansions. 
These  methods  are  limited  to  SDEs  where  the  uncertainties  are  generally  less  than 
10  percent  and  become  extremely  complicated  for  systems  where  there  are  more  than 
two  random  variables.  Moment  Equation  methods  attempt  to  determine  the  moments 
of  the  random  solution  directly  from  the  stated  SDE  problem.  These  methods  are 
difficult  to  use  since  moments  beyond  the  second  moment  (variance  and  covariance) 
are  required  but  most  likely  unknown.  Operator  Methods  attempt  to  express  the  in¬ 
verse  of  the  stochastic  operator  of  the  SDE  as  a  Neumann  series.  Unfortunately,  this 
method  is  also  limited  to  problems  with  small  uncertainties  and  are  not  very  useful. 
Babuska  and  Chatzipantelidis  present  formulation,  analysis,  and  numerical  examples 
of  perturbation  and  operator  methods  applied  to  solve  elliptic  PDEs  in  [3]. 

Another  powerful  technique  is  the  Generalized  Polynomial  Chaos  (gPC)  method. 
The  gPC  is  a  spectral  approximation  of  a  stochastic  process  used  to  solve  SDEs, 
analogous  to  the  deterministic  spectral  methods  described  in  section  2.3.3,  where 
solutions  to  SDEs  are  expressed  as  expansions  of  the  random  parameters  using  or¬ 
thogonal  polynomials.  As  with  deterministic  PSMs,  there  are  both  Galerkin  and 
collocation  implementations  of  the  gPC.  Research,  which  will  be  specifically  cited 
later,  has  shown  that  the  best  choice  of  the  orthogonal  basis  polynomials  depends 
on  the  type  of  PDF  describing  the  random  inputs  and  that  spectral  convergence  is 
achieved  by  selecting  the  appropriate  basis  set.  While  both  gPC  implementations 
are  spectrally  accurate  and  demonstrate  “quick  convergence,”  the  added  benefit  of 
the  collocation  method,  similar  to  Monte-Carlo  Simulation,  is  that  it  is  also  fairly 
simple  to  implement  since  it  only  requires  repeated  deterministic  solutions  at  a  set  of 
collocation  points  [87].  The  gPC  will  be  a  central  concept  in  this  research  and  will 
be  presented  more  thoroughly  in  the  next  subsection. 
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2.4.2  Generalized  Polynomial  Chaos. 


In  1938  Norbert  Wiener  introduced,  for  what  may  be  the  first  time,  a  “Ho¬ 
mogeneous  Chaos”  which  used  Hermite  polynomials  to  approximate  Gaussian  pro¬ 
cesses  [83].  Xiu  summarizes  Wiener’s  work  in  [91]  and  cites  references  to  theorems 
that  have  shown  that  the  Hermite  Chaos  converges  for  any  second-order  random  pro¬ 
cess,  which  are  processes  with  finite  second  moments,  and  in  fact,  converges  spectrally 
for  Gaussian  processes  due  to  the  fact  that  the  Hermite  weighting  function  is  the  same 
as  that  of  the  Gaussian  PDF.  Thus,  Wiener’s  Homogeneous  Chaos,  known  also  as 
Wiener’s  Chaos  or  the  Hermite  Chaos,  enables  approximation  of  second-order  ran¬ 
dom  processes,  which  are  representative  of  many  physical  systems,  using  orthogonal 
polynomials. 

Building  on  cited  work  of  Askey,  who  extended  Wiener’s  Homogeneous  chaos  by 
linking  other  orthogonal  polynomial  bases  and  statistical  distributions  by  matching 
weighting  functions,  Xiu  presented  a  method  for  solving  SDEs  based  on  Galerkin 
projections  that  takes  advantage  of  these  correspondences  [91].  Through  numerical 
experimentation  on  an  ODE  where  an  exact  solution  is  available,  he  demonstrated 
spectral  convergence  of  the  errors  when  these  correspondences  were  followed.  Xiu 
found  that  convergence  was  still  obtained,  but  substantially  slower,  when  the  polyno¬ 
mial  bases  were  not  chosen  to  match  the  probability  distribution.  This  generalization 
of  Wiener’s  Hermite  Chaos  is  known  as  the  gPC  and  a  table  with  the  polynomial 
correlations  to  probability  distributions  is  given  in  the  table  below. 

There  are  two  forms  of  gPC:  Galerkin  and  collocation.  The  Galerkin  form  is 
described  in  [86],  [87],  and  [91]  and  the  procedure  summarized  below  is  a  paraphrase 
from  these  references. 

Consider  the  generic  stochastic  differential  equation,  assuming  that  boundary  con- 


Table  2.3.  Correlation  between  probability  density  functions  and  orthogonal  polynomial 
bases  [88] 


Distribution  Type 

Random  Variable  Distribution 

Polynomial  Basis 

Domain 

Continuous 

Gaussian 

Hermite 

[— oo,  oo] 

Gamma 

Laguerre 

[0,  oo] 

Uniform 

Legendre 

[a,  6] 

Discrete 

Poisson 

Charlier 

0,1,2,... 

Binomial 

Krawtchouk 

0,1, ...,1V 

Negative  Binomial 

Meixner 

0,1,2,... 

Hypergeometric 

Hahn 

0,1, ...,1V 

ditions  are  specified: 


£(x,t,0;u)  =  /(x,t;  9 ) 


(2.100) 


where  u  =  u(x,  t;  9)  is  the  solution  and  /(x,  t;  9)  is  the  source  term.  The  operator  C 
is  a  differentiation  operator  which  can  generally  involve  time  and  space  derivatives 
resulting  in  a  generally  nonlinear  differential  equation  that  can  either  be  an  ODE 
or  PDE  depending  on  the  construction  of  the  operator.  The  random  parameter,  9, 
represents  uncertainty  in  the  system  which  can  be  present  in  the  boundary  conditions, 
initial  conditions,  physical  properties  of  the  system,  etc.  The  presence  of  uncertainty 
causes  the  solution,  u,  to  be  a  random  process,  which  can  be  spectrally  expanded  as 
a  polynomial  chaos  as: 

Pc 

u(x,t;0 )  =  J^Ui(x,f)$?(C(0)))  (2.101) 

2=0 

where  £(0)  is  a  random  variable  (RV)  that  is  a  function  of  the  random  input  6. 
Notice  that  the  sum  has  been  truncated  at  the  P*h  term  and  the  “c”  subscript  is 
used  to  indicate  “chaos”  to  differentiate  this  symbol  from  a  previously  used  one. 
The  set  of  basis  functions,  {$£},  are  orthogonal  polynomials  chosen  based  on  the 
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assumed  distribution  of  the  random  variable  £(•)  as  described  in  Table  2.3.  Again, 
the  superscript  “c”  is  used  to  differentiate  the  “chaos”  expansion  basis  functions 
from  a  similar  formerly  used  symbol.  The  number  of  expansion  terms,  (Pc  +  1),  is 
determined  by  the  dimension,  n,  of  the  random  variable  £  and  the  highest  order,  p, 
of  the  polynomials  as: 


(2.102) 


n\p\ 


Substituting  (2.101)  into  (2.100)  yields: 


(2.103) 


To  minimize  the  approximation  error,  a  Galerkin  projection  is  used  to  project 
(2.103)  onto  each  orthogonal  polynomial  basis  element,  {$^},  using  the  following 
inner  product  definition: 


References  [86],  [87],  and  [91]  show  that  defining  the  inner  product  this  way  ensures 
minimum  approximation  error.  The  result  of  the  projection  is  a  set  of  (Pc  +  1) 
coupled  equations  for  each  iq(x,  t),  where  i  —  0, 1, . . . ,  Pc,  that  are  deterministic  since 
the  uncertainty  has  been  transferred  into  the  polynomial  bases.  However,  the  inner 
products  in  (2.104)  generate  (Pc  + 1)  integral  equations  which  are  solved  numerically 
using  quadrature  rules.  Therefore,  the  time  domain  and  the  space  of  x  need  to 
be  discretized  so  that  deterministic  methods  to  solve  the  integral  equations  can  be 
used,  ft  should  be  obvious  that  solving  SDEs  using  the  Galerkin  form  of  the  gPC 
becomes  increasingly  difficult  as  the  number  of  states  increases.  Further  development 
of  Galerkin  gPC  methods  used  to  solve  elliptic-type  stochastic  PDEs,  most  using 
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Finite  Element  Methods  (FEM)  for  discretization  of  the  physical  space,  can  be  found 
in  [5,23,34,40,90].  A  solution  method  for  hyperbolic  stochastic  PDEs  using  the 
Galerkin  gPC  method,  with  application  to  a  wave  equation  example,  is  presented 
in  [45]. 

The  Galerkin  formulation  of  the  gPC  has  been  discussed  and  now  the  collocation 
form  will  be  presented.  As  has  been  stated,  the  gPC  is  a  spectral  approximation 
of  stochastic  process,  which  is  analogous  to  the  discussion  of  spectral  methods  in 
section  2.3.3.  Detailed  development  of  the  collocation  type  gPC  for  solving  stochastic 
PDEs  and  ODEs  can  be  found  in  [4,  25,  89]  and  rigorous  convergence  analysis  is 
performed  in  [4,66].  Pulch  adapts  the  collocation  gPC  to  solve  BVPs  in  [70],  noting 
that  other  numerical  methods  like  shooting,  finite  difference,  and  spectral  expansion 
methods  described  in  section  2.3  have  analogous  applications  to  stochastic  ODEs.  The 
focus  here  will  be  placed  on  the  approach  formulated  to  solve  ODEs  and  Differential 
Algebraic  Equation  (DAE)s,  which  are  the  most  commonly  seen  types  of  equations 
in  OC  problems  and  the  types  addressed  in  this  research  effort.  More  details  are 
presented  than  in  the  Galerkin  case  since  the  collocation  formulation  will  be  used  in 
this  research.  The  development  of  the  key  equations  of  the  collocation  gPC  solution 
process  are  paraphrased  from  [86] .  The  author’s  notation  has  been  preserved  as  much 
as  possible  since  it  appears  to  be  fairly  standard  in  the  literature.  The  step-by-step 
procedure  combining  the  GPM  and  collocation  form  of  the  gPC  used  in  this  research 
will  be  concisely  presented  in  Chapter  Ill  and  application  of  the  algorithm  to  two 
types  of  optimization  problems,  an  OC  problem  with  state  uncertainties  and  a  TO 
problem  with  uncertainties  effecting  the  cost  functional,  will  be  presented  in  Chapters 
IV  and  V,  respectively. 

The  collocation  form  gPC  algorithm  development  will  consider  a  general  system 
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of  DAEs  in  the  following  form: 


0 


t  ^  (to,  T } 


(2.105) 


s(*o,y(fo),-,y(0(fo),p)  =o 


where  the  key  variables  are: 

•  State  variables:  y  =  (yi,...,yj)  G  MJ 

•  Stochastic  input  parameters  (random  variables):  p  =  (pi,...,pjv)  G 

•  Outputs  or  observables:  z  =  G( y)  =  (zi, ... ,Zk )  £  KA.  Generally  speaking, 
the  outputs  or  observables  can  be  the  states  or  some  function  of  the  states 
that  defines  quantities  of  interest.  When  applied  specifically  to  optimization 
problems,  the  observables  can  be  the  states,  costates,  controls,  cost,  terminal 
time  (free- final-time  problem),  Hamiltonian,  and  so  on. 

Additionally,  its  necessary  to  define  the  probabilistic  structure.  The  vector  p  is 
defined  as  an  N-variate  random  vector  whose  elements  are  assumed  to  be  independent 
of  each  other.  If  the  random  elements  in  p  are  not  independent,  then  a  transformation, 
such  as  the  Karhunen-Loeve  decomposition  or  Rosenblatt  transformation,  should  first 
be  applied  resulting  in  a  set  of  uncorrelated  RVs  that  are  assumed  to  be  independent 
[11,39,87,89,92],  Furthermore,  assume  that  the  probability  space  is  defined  by  the 
tuple  (Q,A,V),  where  the  space  of  all  possible  basic  outcomes  is  fl,  is  equipped 
with  cr-algebra  A,  which  can  be  thought  of  as  the  space  of  possible  events  that  can 
be  derived  from  H,  and  probability  measure  V  [40].  Now  define  key  probabilistic 
variables  in  the  gPC  collocation  development: 

•  PDF:  Pi  is  the  PDF  of  the  random  variable  Pi(oj),  for  %  —  1, ...,  N  and  a j  £  fl. 
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Because  the  elements  of  p  are  independent,  the  joint  PDF  is  given  by: 


N 

p(p)=n^)  (2.106) 

i—  1 

•  Finite  domain  of  pf  P,;  =  transforms  the  infinite  domain  to  the  hnite 
domain  T  which  are  intervals  in  R,  for  i  —  1,  The  total  hnite  domain  is 

defined  as: 

N 

P.n^M"  (2.107) 

2=1 

One  goal  of  the  gPC  expansion  is  to  approximate  the  RVs  in  p  using  orthogonal 
polynomials.  The  one- dimensional  orthogonal  polynomial  space,  Wl,di,  of  highest 
degree,  di,  is  defined  by: 


=  {v  .  r.  M  :  v  G  spanifafa)}**}  (i  =  1,  N)  (2.108) 


The  one-dimensional  polynomial  basis  set,  {4>miPi)}i  which  has  di  +  1  basis  elements 
and  chosen  according  to  Table  2.3,  satisfies  the  following  orthogonality  conditions: 


Pi&iWmtPiWniV^dVi 


hm5„ 


(2.109) 


where  the  polynomial  basis  elements  are  made  orthonormal  with  respect  to  the  PDF 
by  applying  appropriate  scaling  factors  such  that  h‘2m  =  1  for  all  m  [87]. 

The  N-variate  space  of  orthogonal  polynomials,  W^c,  of  total  degree  of  at  most 
Pc,  is  constructed  by  taking  the  tensor  product,  i.e.  all  possible  combinations,  of  the 
N  one-dimensional  orthogonal  polynomial  bases,  written  as: 


=  (g)  WiA 

|d|  <PC 


(2.110) 
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where  the  relationship  between  di  and  Pc  is  given  by: 

N 

|d  \  =  '52di<Pc  (2.111) 

i= 1 

and  satisfy  the  orthogonality  conditions: 


J  ®Cm(p)K(p)p(p)dP  =  E  [$m(p)$n(p)]  =  dmn  (2.112) 


The  effect  of  (2.111)  is  to  select  a  subset  of  basis  elements  from  the  tensor  product 
space  defined  by  (2.110)  in  order  to  reduce  computational  burden,  however  the  entire 
set  of  basis  elements  resulting  from  the  tensor  product  can  be  used  [87],  in  which  case 
M  is  equal  to  the  total  number  of  basis  elements  resulting  from  the  tensor  product 
of  the  one-dimensional  basis  sets. 

Now  the  approximation  of  the  output,  z,  as  a  function  of  the  random  parameters, 
p,  can  be  written  as: 

M 

Fvz  =  zv(p)  =  J]  zm$m( P)  (2-113) 

m—  1 


where: 


M 


m 


(2.114) 


or  is  the  total  number  of  tensor  product  elements  if  the  entire  set  of  basis  elements  are 
being  used.  The  projection  operator  in  (2.113)  ,  Pjy,  projects  the  discrete  probability 
space  T  onto  the  polynomial  space  Wjf.  The  coefficients,  zm,  are  determined  by: 


zm  =  E  [z(p)$^(p)]  =  j  z(p)<&cm{p)p(p)dp  (m  =  1, ...,  M)  (2.115) 

The  key  element  in  the  gPC  collocation  method  is  to  approximate  the  integral 
in  (2.115)  using  a  Gaussian  quadrature  rule.  To  apply  quadrature,  a  set  of  colloca- 
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tion  points  must  be  chosen  and  associated  quadrature  weights  calculated.  For  each 
random  dimension,  Tj,  i  =  1, . . . ,  N,  collocation  nodes  and  weights  are  chosen  to 
correspond  with  the  polynomial  representation  described  in  Table  2.3  and  determined 
by  information  given  in  Table  2.2.  The  N-dimensional  grid  of  Q  collocation  nodes 
and  weights  is  obtained  by  taking  the  tensor  product  of  the  one-dimensional  node  and 
weight  sets.  Alternatively,  a  sparse  grid  can  be  generated  to  reduce  the  total  number 
of  tensor  product  nodes  while  providing  accurate  integral  approximation  using  the 
Smolyak  algorithm,  first  published  in  [77]  and  included,  along  with  other  sparse  grid 
schemes,  in  [38].  In  general,  a  tensor  product  grid  works  well  for  low-dimensional 
problems,  but  suffers  the  curse  of  dimensionality  as  N  gets  larger  (N  >  5)  making 
sparse  girds  a  better  choice  [4,87].  The  following  figure  shows  representative  two- 
dimensional  ( N  =  2)  collocation  grids  generated  using  both  schemes.  The  tensor 
product  grid  was  generated  using  20  LG  points  in  each  dimension  resulting  in  a  total 
of  Q  =  400  collocation  points  and  quadrature  weights.  The  sparse  grid  was  generated 
using  a  MATLAB®routine  [10,11]  that  generated  nodes  and  weights  associated  with  a 
fifth-order  quadrature  rule,  resulting  in  Q  =  290  quadrature  points  and  weights.  The 


Figure  2.1.  Collocation  grids  used  in  gPC  collocation  expansion.  Left:  Two-dimensional 
(N  =  2)  tensor  product  collocation  grid  with  20  LG  points  (qi  =  20)  in  each  dimension. 
Right:  Sparse  collocation  grid  in  two-dimensions  based  on  5th-order  quadrature  rule. 
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collocation  approximation  to  (2.115)  based  on  the  quadrature  rule  chosen  is  given  by: 


Q 

Zm  =  ^2z{pj)$cm(pj)aj]  (m  =  1, M)  (2.116) 

3  = 1 

where  p7  denotes  the  jth  collocation  node  of  the  random  vector  p,,  =  (pij,  ...,pN,j) 
and  ctj  denotes  the  associated  quadrature  weight,  thus  defining  the  set  of  collocation 
nodes  and  weights,  {p j,ctj}®=1.  Note  that  z(pj)  is  the  deterministic  solution  using 
the  jth  sample  of  the  random  vector.  This  gPC  formulation  is  the  basis  of  a  general 
procedure  which  will  be  presented  in  Chapter  III. 

It  can  be  seen  that  one  advantage  of  the  gPC  collocation  method,  elegantly  stated 
in  [62],  which  also  presents  the  gPC  collocation  algorithm  development,  is: 

...while  the  standard  Galerkin  approach  to  Polynomial  Chaos  requires 
multi- dimensional  summations  over  the  stochastic  basis  functions,  the 
stochastic  collocation  method  enables  to  collapse  those  summations  to 
a  one-dimensional  summation  only. 

In  other  words,  a  one-dimensional  summation  replaces  multi-dimensional  summations 
used  in  the  Galerkin  method  to  numerically  approximate  the  integrals  resulting  from 
the  inner  products.  Mathelin  demonstrates  this  through  formulation  of  a  solution  to 
a  one-dimensional  nozzle  flow  (fluids)  problem  where  the  Galerkin  method  results  in 
a  nine- dimensional  summation,  or  more  depending  on  boundary  conditions.  Other 
advantages  to  the  gPC  collocation  method,  as  stated  in  [4, 87]  are: 

•  The  gPC  collocation  leads  to  solution  of  uncoupled  deterministic  problems 
where  the  gPC  expansion  is  a  post-processing  step  after  the  deterministic  solu¬ 
tions  at  the  collocation  nodes  have  been  found. 

•  The  gPC  can  handle  unbounded  random  variables,  for  example  Gaussian  or 
exponentially  distributed  random  variables. 
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•  Like  the  Galerkin  form  of  the  gPC,  it  is  spectrally  accurate,  i.e.  the  algorithm 
gives  exponential  convergence. 

The  gPC  algorithms  have  been  applied  to  solve  SDEs  in  many  engineering  disci¬ 
plines  such  as:  analysis  of  nonlinear  integrated  circuit  response  [80],  various  types  of 
fluids  problems  [45,57,58,62,92],  heat  transfer  [41,82,93],  and  others  referred  to  by 
the  authors  of  these  references.  Applications  of  the  gPC  methods  applied  to  stochas¬ 
tic  OC  problems  are  rare.  One  such  application  is  found  in  [11]  where  a  nonlinear 
parabolic  PDE-based  OC  problem  is  considered  using  “multi-grid”  physical  domain 
discretization.  The  difference  between  the  work  in  this  reference  and  the  research 
developed  in  this  document  is  that  the  former  is  not  using  a  spectral  method  to  solve 
the  deterministic  problems  at  the  discretized  random  sample  points.  No  references 
containing  work  similar  to  that  described  in  this  document  have  been  found  thus  far. 

2.4.3  Numerical  Methods  for  Stochastic  Differential  Equations  Sum¬ 
mary. 

This  section  began  with  a  survey  of  commonly  used  techniques  to  solve  stochastic 
optimal  control  problems.  The  discussion  was  then  expanded  to  consider  methods 
used  to  solve  stochastic  differential  equations,  which  may  be  adapted  to  solve  stochas¬ 
tic  optimal  control  problems.  Specifically,  Generalized  Polynomial  Chaos  methods 
were  presented  since  they  provide  spectral  representations  of  the  random  space  that 
are  analogous  to  the  spectral  representations  of  deterministic  problems  discussed  ear¬ 
lier  in  the  document  and  provide  accurate  approximations  to  the  solution.  Two  forms 
of  the  Generalized  Polynomial  Chaos  were  summarized:  Galerkin  and  Collocation. 
Again,  parallels  between  the  deterministic  and  stochastic  versions  of  these  algorithms 
are  obvious,  where  the  collocation  approach  is  easier  to  implement  while  providing 
accuracy  that  is  similar  to  the  Galerkin  method.  The  abbreviated  version  of  the 
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collocation  method  was  given  at  the  end  of  the  section,  which  will  be  adapted  and 
applied  to  a  sample  stochastic  optimal  control  problem  in  the  next  chapter. 

2.5  Related  Concepts  Summary 

This  chapter  began  with  a  brief  exposition  on  the  historical  evolution  of  OC 
theory.  The  general  formulation  of  a  continuous  OC  problem  was  posed  along  with 
the  traditional  solution  technique  applying  CV  theory  and  the  PMP  to  transform 
the  problem  into  an  HBVP.  Recognizing  that  analytically  solving  the  resulting  set 
of  coupled  nonlinear  differential  equations  defining  first-order  optimality  conditions 
may  be  impossible,  the  formulation  provided  motivation  for  discussing  indirect  and 
direct  numerical  solution  methods. 

Indirect  methods  seek  to  numerically  solve  the  system  of  ODEs  comprising  the 
HBVP.  As  such,  discussion  of  indirect  methods  presented  focused  primarily  on  numer¬ 
ical  integration  rules  as  implemented  in  the  time-marching,  shooting,  and  collocation 
methods.  Direct  methods,  on  the  other  hand,  avoid  analytical  derivation  of  the  nec¬ 
essary  conditions  for  optimality  like  those  in  the  HBVP  by  transcribing  the  original 
problem  into  a  parameter  optimization  problem  and  applying  NLP  techniques.  Basic 
concepts  of  formulating  an  NLP  problem  were  presented  and  how  direct  shooting  and 
collocation  methods  are  cast  as  NLP  problems. 

Collocation  and  NLP  concepts  were  then  put  together  to  form  the  basis  for  PSMs, 
which  can  be  applied  to  both  indirect  HBVP  and  the  direct  NLP  problem  formula¬ 
tions.  After  general  discussion  of  spectral  methods  highlighting  basic  concepts  and 
available  forms,  particular  attention  was  paid  to  the  direct  Gauss  Pseudospectral 
Method  where  equations  transcribing  the  optimal  control  problem  to  nonlinear  pro¬ 
gramming  problem,  Karush-Kuhn-Tucker  optimality  conditions,  equivalence  of  GPM 
and  HBVP  solutions,  and  GPM  implementation  in  GPOPS  were  discussed. 
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Emphasis  was  then  shifted  to  address  stochastic  solution  methods.  A  summary 
of  commonly  used  techniques,  most  notably  MCS,  to  quantify  uncertainty  in  SDEs 
ultimately  led  to  presentation  of  the  collocation  form  of  the  gPC,  which  is  a  spectral 
method  analogous  to  the  deterministic  spectral  methods  presented  in  the  chapter. 
The  gPC  uses  orthogonal  polynomial  bases,  chosen  based  on  correlation  between 
weighting  functions  of  the  assumed  PDFs  of  the  uncertain  parameters  and  the  poly¬ 
nomial  bases,  to  approximate  SDE  solutions  as  functions  of  the  uncertain  parame¬ 
ters.  The  collocation  form  applies  deterministic  solution  techniques  at  the  collocation 
points  to  generate  a  set  of  deterministic  solutions  used  in  stochastic  computations. 
Therefore,  it  seems  possible  to  use  GPOPS  as  the  deterministic  solver  implement¬ 
ing  the  GPM  to  provide  spectrally  accurate  approximations  to  OC  problems,  which 
have  been  shown  to  be  equivalent  to  numerical  solutions  of  the  CV-based  HBVP,  in 
the  gPC  construct  to  extend  the  method  to  be  applicable  to  stochastic  optimization 
problems.  Combining  the  GPM  and  gPC  spectral  methods  into  a  hybrid  algorithm 
will  be  presented  in  the  next  chapter. 
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III.  Hybrid  GPM-gPC  Algorithm 


The  focus  of  this  research  is  to  determine  if  a  method  that  integrates  the  GPM 
discussed  in  section  2. 3. 3. 2  into  the  collocation  form  of  the  gPC  discussed  in 
section  2.4.2  is  capable  of  solving  nonlinear  optimization  problems  by  quantifying 
the  effects  random  parameters  have  on  the  solutions.  Using  the  GPM  numerical 
solution  package  GPOPS  in  place  of  the  differential  equation  solvers  traditionally 
used  in  the  gPC  construct  will  provide  a  set  of  minimum  cost  numerical  solutions 
to  the  sampled  optimization  problems  that  satisfy  boundary,  path,  and  bounded 
state  and  control  constraints.  Since  these  solutions  are  spectrally  accurate  and  have 
been  shown  to  be  equivalent  to  numerical  solutions  of  the  HBVP,  they  are  the  best 
numerical  approximations  available  to  the  true  solution  and  are  suitable  for  stochastic 
computations. 

The  algorithm  presented  in  this  chapter,  as  shown  in  figure  3.1  and  discussed  in 
the  process  steps,  is  an  adaptation  of  the  gPC  collocation  algorithm  published  in  [86]. 
The  gPC  algorithm  presented  in  the  reference  was  originally  written  to  solve  SDEs 
by  applying  IVP  and  BVP  differential  equation  routines  to  generate  the  determin¬ 
istic  solution  set  that  is  used  in  stochastic  computations.  Xiu’s  algorithm  has  been 
adapted  to  form  a  hybrid  algorithm  that  is  applicable  to  OC  and  TO  problems  by 
using  GPOPS  in  place  of  the  IVP  and  BVP  solvers  [86].  The  remainder  of  this  docu¬ 
ment  will  be  focused  on  application  of  the  hybrid  algorithm  combining  the  GPM  and 
gPC  collocation  methods.  For  convenience,  future  use  of  the  gPC  abbreviation  refers 
to  the  collocation  form  of  the  gPC  and  the  term  hybrid  algorithm  refers  to  the  col¬ 
location  form  of  the  gPC  where  the  GPM  tool,  GPOPS,  is  used  as  the  deterministic 
OC  solver  in  the  gPC  collocation  construct.  The  hybrid  algorithm  is  discussed  in  the 
following  five  steps. 
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Step  #1:  Choose  a  set  of  collocation  nodes  and  quadrature  weights,  {p j,aj}f=1,  in 
the  finite  probability  space,  T. 

Since  the  hybrid  algorithm  will  generate  a  spectral  approximation  of  the  solution 
to  the  stochastic  problem,  points  that  match  both  the  quadrature  rule  evaluating 
the  expansion  coefficients  given  by  (2.115)  and  the  polynomial  basis  elements 
should  be  used.  For  example,  if  Hermite  orthogonal  polynomial  bases  are  chosen  to 
approximate  a  Gaussian  distribution  as  suggested  in  [87],  [88],  and  Table  2.3,  then 
Gauss-Hermite  quadrature  is  an  appropriate  integration  rule  and  will  be  most  ac¬ 
curate  using  Gauss-Hermite  nodes  and  weights.  Furthermore,  the  Gauss-Hermite 
quadrature  nodes  will  define  the  finite  random  domain  T.  The  Gaussian  distribution 
is  supported  on  fl  =  (— oo,  oo)  with  an  infinite  number  of  possible  realizations,  but 
when  the  quadrature  points  are  chosen,  V  will  be  supported  on  some  bounded  region 
in  M  with  a  finite  number  of  points.  The  grid  of  Q  points  is  constructed  by  choosing  g* 
collocation  nodes  and  quadrature  weights  in  each  random  domain,  Tj,  for  %  =  1, . . . ,  N, 
and  then  taking  the  tensor  product  of  the  N  individual  sets.  As  mentioned  in  Chap¬ 
ter  II,  there  is  an  abundance  of  MATLAB®functions  available  for  download  on  the 
MathWorks  website  to  generate  quadrature  nodes  and  weights.  Descriptions  of 
other  quadrature  point  sets  and  rules  can  be  found  in  [14],  [33],  and  [81].  Each  node 
in  the  grid  is  used  as  an  input  to  the  governing  stochastic  OC  problem  equations 
resulting  in  a  set  of  deterministic  problems  that  are  solved  in  Step  ^2  with  existing 
numerical  tools  like  GPOPS. 

Step  #2:  Using  each  of  the  Q  sample  points,  Pj,  solve  the  set  of  deterministic  OC 
problems  using  GPOPS  and  build  the  observables  array  z  =  G(x),  consisting  of  state 
trajectories,  control  solutions,  and  cost  values  as  well  as  any  other  variables  of  interest 
such  as  costate  trajectories,  Hamiltonain  values,  terminal  time,  or  any  other  variables 
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available  in  the  GPOPS  solution  data. 


Using  the  GPM  to  solve  the  Q  deterministic  OC  problems  at  the  collocation 
points  is  one  reason  why  this  hybrid  algorithm  is  unique.  Reference  [86]  states  that 
any  deterministic  solver  can  be  used,  which  for  the  author’s  purposes  was  an  IVP 
solver.  MATLAB®provides  effective  solvers,  such  as  ODE^5  for  I  VPs  and  BVP^c  for 
BVPs,  that  could  also  be  used  to  solve  the  deterministic  set  of  HBVP  differential 
equations  at  the  collocation  points.  However,  using  the  direct  GPM  method  coded  in 
the  GPOPS  software  provides  spectrally  accurate  solutions  to  the  deterministic  sam¬ 
pled  problems,  if  they  exist,  that  are  equivalent  to  numerical  solutions  of  the  HBVP 
differential  equations,  satisfy  optimality  conditions,  minimize  the  performance  index, 
and  satisfy  the  boundary  conditions  while  avoiding  complexities  of  explicitly  deriving 
and  solving  the  HBVP  differential  equations  mentioned  in  sections  2.2.2  and  2.3.1. 
Therefore,  GPOPS,  or  similar  PSM  solvers,  provides  the  best  possible  approximations 
to  the  Q  deterministic  problems  whose  solutions  are  used  in  Step  #3  to  calculate 
the  expansion  coefficients. 

Step  #3:  Evaluate  the  expansion  coefficients  using  (2.116),  which  is  restated  below: 

Q 

Zm  =  5Zz(pj)$m(pj)ai;  =  (3-1) 

3  = 1 

These  expansion  coefficients  contain  the  information  necessary  to  approximate  the 
statistical  properties  of  the  solution  constructed  in  Step  #4. 

Step  #4:  Build  the  output  approximation  function  using  (2.113),  which  is  restated 
below: 

M 

PAZ  =  ZN  (P)  =  X]  Zm$m(P)  (3.2) 

m=  1 
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The  solution  to  the  stochastic  problem  given  by  (3.2)  is  essentially  a  set  of  distri¬ 
bution  functions  that  estimate  the  outputs  (observables)  as  functions  of  the  random 
inputs,  the  statistics  of  which  are  determined  in  Step  #5. 


Step  #5:  Evaluate  the  statistics  of  the  approximate  solution. 

The  statistical  information  about  the  solution  is  the  desired  output  and  is  easily 
calculated  using  the  expansion  coefficients.  Expected  value,  variance,  and  covariance 
functions  describe  the  most  likely  solution  to  the  stochastic  problem,  characterizes 
how  the  solution  varies  with  random  inputs,  and  describes  the  dependencies  between 
the  outputs,  respectively.  Equations  for  these  properties  are  derived  in  [87]  and  are 
stated  below. 

The  expected  value  (E)  is: 


E(z(t))  «E(z£(t)) 


M 


y  zm(t)$m(p) 


m= 1 


p(p)dp  =  zt(t) 


(3.3) 


The  variance  ( var )  is: 


M 

var  [z (t)]  «  y  [z 2m(t)] 

m= 2 


(3.4) 


The  covariance  ( cov )  is: 


M 

cov  [ Zi(t),Zj(t)\  fa  £  »,„(«)%„(*)]  (3.5) 

m= 2 

This  process  was  implemented  in  MATLAB®and  used  to  investigate  an  optimal 
control  problem  with  state  uncertainties  in  Chapter  IV  and  a  trajectory  optimiza¬ 
tion  problem  with  constraint  uncertainties  in  Chapter  V  with  the  following  research 
questions  in  mind. 


•  How  can  an  OC  or  TO  problem  with  uncertain  parameters  be  formulated  so 
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that  it  can  be  solved  using  the  hybrid  GPM-gPC  algorithm? 

•  Can  the  algorithm  be  applied  to  problems  with  various  types  of  boundary  con¬ 
ditions  and  constraints? 

•  Can  the  algorithm  be  applied  to  problems  where  uncertain  parameters  effect 
different  equations? 

•  How  do  the  hybrid  algorithm  solutions  compare  with  MCS  solutions? 

•  Can  the  hybrid  algorithm  solve  problems  where  the  uncertain  parameters  do 
not  all  have  the  same  assumed  PDF  and  thus  use  a  mix  of  polynomial  bases? 

•  Can  the  hybrid  algorithm  be  applied  to  real-world  scenarios  to  provide  useful 
information  to  the  users? 

This  concise  chapter  has  served  as  a  bridge  between  the  GPM  and  gPC  theoretical 
concepts  presented  in  sections  2. 3. 3. 2  and  2.4.2  and  their  implementation  into  the 
hybrid  algorithm  shown  in  Figure  3.1  and  discussed  in  the  step-by-step  process  listed. 
Several  research  questions  have  also  been  posed  that  will  be  investigated  by  applying 
the  hybrid  GPM-gPC  algorithm  to  two  types  of  optimization  problems  in  the  following 
two  chapters. 


84 


Figure  3.1.  Hybrid  algorithm  combining  GPM  and  gPC  methods. 
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IV.  Optimal  Control  Problem 


In  this  chapter,  the  hybrid  numerical  algorithm  combining  the  GPM  and  gPC 
methods  outlined  in  Chapter  III  is  applied  to  an  OC  problem  that  has  nonlinear 
dynamics  and  performance  index  with  multiplicative  uncertainties  in  the  state  equa¬ 
tions.  The  objective  of  this  problem  is  to  demonstrate  the  hybrid  algorithm’s  ability 
to  quantify  the  effects  uncertain  parameters  have  on  OC  solutions  by  applying  it  to 
a  challenging  nonlinear  problem. 

4.1  Problem  Formulation 

The  OC  concept  demonstration  problem  is  an  adaptation  of  problem  5-2  in  [54], 
This  deterministic  baseline  problem  will  be  stated  first  in  section  4.1.1  followed  by 
modifications  that  were  used  to  create  the  stochastic  problem  variant  described  in 
section  4.1.2.  The  stochastic  problem  is  further  developed  following  the  gPC  col¬ 
location  modeling  in  section  2.4.2  resulting  in  a  formulation  that  can  be  solved  by 
applying  the  hybrid  algorithm.  The  solution  to  the  deterministic  problem  is  included 
with  the  hybrid  algorithm  results  in  section  4.2  to  compare  with  the  expected  value 
solution  as  a  check  that  the  hybrid  algorithm  solution  is  reasonable. 

4.1.1  Deterministic  Baseline  Problem. 

The  deterministic  baseline  problem  is  stated  as: 

•  Find  u[t)  G  R1  and  x(t)  £  M2  that  minimizes  the  cost  functional  ( J): 

J  =  J  ^[2aa(t)  +  x\  (t)  +  u2(t)]dt 
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(4.1) 


•  subject  to: 


Xi  (t)  =  x2{t)  (4.2) 

x2 (t)  =  —xi(t)  +  [1  -  x\(t)\x2(t)  +  u(t)  (4.3) 

•  with  boundary  conditions: 

x(0)  =  [0,0]T  (4.4) 

x(l)  =  [1,10]T 

GPOPS  automatically  formats  the  problem  to  be  in  the  form  of  (2.69)  -  (2.72)  by 
transforming  t  £  [0,1]  to  r  e  [— 1, 1]  using  (2.73).  This  is  a  nonlinear,  continuous¬ 
time,  Lagrange-type  OC  problem  without  path  and  control  constraints,  and  with 
fixed  final  state  and  time.  It  can  be  verified  that  forming  the  HBVP  as  described 
in  section  2.2.2  results  in  a  set  of  coupled  ODEs  that  cannot  be  solved  analytically, 
making  it  well  suited  for  direct  numerical  solution  in  GPOPS. 

4.1.2  Stochastic  Problem. 

Uncertainty  is  incorporated  by  modifying  the  dynamics  (4.2)  as  indicated  by  the 
(—>■),  with  random  input  parameters  A  and  B  as  follows: 

xi{t)  — >■  xi(t)  +  Axi(t)  =  (1  +  A)xi(t)  (4.5) 

x2(t)  ->■  x2(t)  +  Bx2(t)  =  (1  +  B)x2(t) 
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resulting  in  the  modified  stochastic  dynamics: 


x\  (t)  =  x2(t)  +  Bx2(t)  (4.6) 

x2(t)  =  —(xi(t)  +  Axi(t))  +  u(t) 

+  [1  -  (xi(t)  +  ^Xi(t))2](x2(t)  +  Bx2(t )) 


Inputs  A  and  B  are  assumed  to  be  independent  Gaussian  RVs  and  come  from  the 
same  distribution  with  zero  mean,  /i  =  0,  and  standard  deviation,  a,  of  0.1.  These 
random  inputs  introduce  uncertainties  on  the  states  transforming  x(f),  u(t),  and  J 
into  RVs.  The  cost  functional  (4.1),  boundary  conditions  (4.4),  and  time  domain 
remain  the  same  in  the  stochastic  problem.  It  should  be  noted  that  the  objective  of 
a  stochastic  OC  problem  is  typically  to  find  the  control  signal  and  state  trajectories 
that  minimize  the  expected  value  of  the  cost  functional,  written  as: 


J 


E 


(2  x\(t)  +  x\(t)  +  u2(t))dt 


(4.7) 


In  this  case,  quantification  of  the  effects  of  uncertainties  on  x(£),  u(t),  and  J(x,  u) 
solutions  is  sought  since  it  is  assumed  that  there  is  no  way  to  control  the  uncertainties 
or  minimize  their  effects.  So,  the  cost  functional  as  written  in  (4.1)  is  used  when 
solving  each  of  the  Q  sampled  deterministic  problems  and  the  set  of  solutions  is  used 
to  construct  gPC  approximations  as  functions  of  the  random  inputs  to  assess  the 
effects  of  those  uncertainties. 

Continuing  with  the  problem  set-up,  the  following  definitions  are  used  following 
the  development  in  section  2.4.2: 

•  State  variables:  x  =  (x\,x2)  G  M2  ( J  —  n  —  2) 

•  Stochastic  input  parameters  (random  variables):  p  =  (A,  B)  G  M2 

(N  =  2). 
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•  Outputs  or  observables:  the  observables  are  chosen  to  be  the  state  variables, 
control,  and  cost  values,  z  =  (x\ ,x2,u,  J )  G  M4  ( K  =  4) 

The  PDFs  (Gaussian)  of  A  and  B  are  denoted  by  pi(A)  and  P2^B).  The  joint 
PDF  is  given  by  (2.106)  as: 

2 

p(p) = n*(*) = pi(a)pi(b)  (4-g) 

i= 1 

The  sample  space,  D,  is  (— oo,  oo)  x  (—00,00),  but  choosing  the  distributions  with 
Pa  =  Pb  =  0  and  crA  =  a B  =  0.1  results  in  A  and  B  taking  values  in  a  10cr  window, 
(— 1, 1)  x  (—1, 1),  with  greater  than  99.99%  probability.  Defining  the  distributions  in 
this  way  avoids  non-existent  OC  problem  solutions  resulting  from  state  cancellation 
in  the  dynamics  as  A  and  B  approach  —1. 

The  collocation  points  define  the  finite  sample  space.  Since  Gaussian  PDFs  are 
assumed,  Hermite  polynomial  basis  functions  were  chosen  for  the  gPC  expansions,  as 
suggested  in  [87],  [88],  and  Table  2.3,  and  collocated  at  Gauss-Hermite  quadrature 
points.  It’s  important  to  note  that  the  statistical  version  of  Hermite  polynomials  is 
used  where  the  weighting  function  is  exp[—  ]  instead  of  the  typical  exp[—x2], 
implying  that  the  traditional  Gauss-Hermite  quadrature  points  should  be  scaled  by 
\/2<72  and  shifted  by  adding  p  and  the  quadrature  weights  scaled  by  (-y/F)^1.  Recall 
there  are  q  points  in  each  random  dimension  calculated  by  finding  the  roots  of  the 
qth  Hermite  polynomial,  Hq ,  and  scaled  accordingly.  For  this  sample  problem,  qi  and 
q2  were  chosen  to  be  21,  resulting  in  Q  =  441  collocation  nodes  shown  in  Fig.  4.1. 
Figure  4.1  shows  that  the  resulting  finite  probability  space,  V  =  ]Qi=i  as  given  in 
(2.107),  is  the  tensor  grid  [—0.8,  0.8]  x  [—0.8,8]. 

The  one-dimensional  polynomial  spaces  (2.108),  were  chosen  to  be  represented  by 
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Figure  4.1.  Finite  domain,  Ti  x  lb,  defined  by  the  tensor  grid  of  441  collocation  points 
used  as  random  inputs  A  and  B. 

the  following  polynomial  basis  sets: 

{«*.»  =  (4.9) 

{«*=o  = 

where  Hi  denotes  the  Hermite  polynomial  of  the  ith  order.  Thus,  the  2-dimensional 
polynomial  space,  defined  using  the  tensor  product  in  (2.110),  is: 

<“  =  {«*,  o(g){02}Sa_o  (4-10) 

Selecting  the  entire  set  of  basis  functions  yields  36  total  basis  elements,  where  each 
one- dimensional  basis  polynomial  is  fifth  order  and  the  order  of  the  tensor  product, 
Pc,  is  10.  The  number  of  basis  elements  was  chosen  to  provide  balance  between 
computational  burden  and  lowest  observed  variances  in  the  state  approximations  at 
the  terminal  time,  indicating  satisfaction  of  specified  terminal  boundary  conditions. 
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4.2  Results 


Results  of  applying  the  hybrid  GPM-gPC  algorithm  to  the  OC  demonstration 
problem  are  presented  in  this  section.  Mean  solutions  and  variances  are  shown  for 
the  state  variables,  x\ (t),  x2(t),  and  control,  u(t),  as  well  as  the  covariances  be¬ 
tween  these  variables,  and  are  compared  to  an  MCS  for  verification.  The  variation  of 
J(x(t),  u(t),  t;  A,  B )  with  respect  to  the  random  inputs  is  also  shown. 

The  MCS  was  accomplished  by  generating  random  samples  of  the  stochastic  vari¬ 
ables,  [A,  B),  taken  from  Gaussian  distributions  with  ha  =  Hb  =  0  and  <ja  =  o~b  = 
0.1  as  specified  in  the  problem  formulation.  Each  sample  pair  was  subsequently  in¬ 
serted  into  the  stochastic  dynamics  equations  (4.6),  creating  a  set  of  deterministic 
OC  problems  that  were  solved  using  GPOPS.  The  ensemble  of  solutions  was  used  to 
calculate  expected  values  (mean)  and  variances  of  the  state  variables  and  control  and 
covariances  between  x\ (t)  and  u(t),  x2(t)  and  u(t),  and  Xi(t)  and  x2(t).  Convergence 
was  checked  after  every  500  samples  and  was  achieved  at  16,  500  samples  when  the 
maximum  change  in  expected  values  of  the  state  trajectories  from  the  previous  inter¬ 
val  was  below  an  arbitrarily  chosen  tolerance  of  5  x  10-5.  Expected  value,  variance, 
and  covariance  properties  of  the  hybrid  algorithm  solution  are  compared  to  MCS 
results  using  the  following  Percent  Difference  (PD)  equation: 


PD  =  max 


SMCSjt)  —  Sglgjt ) 

( SMCs(t )  +  Saig(t))/ 2 


x  100 


(4.11) 


where  hybrid  algorithm  and  MCS  statistical  quantities  being  compared,  Saig  and 
SmcSi  are  the  expected  values,  variances,  or  covariances  as  appropriate.  This  com¬ 
parison  metric  was  chosen  over  the  typical  percent  error  because  both  hybrid  algo¬ 
rithm  and  MCS  results  are  approximations  where  fidelity  can  be  increased,  at  the 
expense  of  computational  burden,  by  increasing  the  numbers  of  sample  points  or  by 
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Figure  4.2.  Hybrid  algorithm,  MCS,  and  deterministic  expected  value  estimates  of 

x\ (t),  X2 (t),  and  w(i). 

increasing  the  polynomial  order,  and  thus  neither  can  be  assumed  to  be  a  theoretical 
answer. 

Figure  4.2  shows  the  mean  solutions  for  x\ (t),  X2 (i),  and  u(t),  respectively.  The 
three  curves  on  each  of  these  plots  show  the  hybrid  algorithm  expected  value  solution, 
the  MCS  expected  value  solution,  and  the  deterministic  solution.  The  hybrid  algo¬ 
rithm  expected  value  approximation  for  each  state  and  control  variable  is  given  by 
the  first  coefficient  in  each  expansion  as  stated  in  (3.3).  Bars  indicating  the  standard 
deviations  are  included  to  quantify  how  much  the  solution  varies  from  the  mean.  The 
solution  to  the  deterministic  OC  problem,  given  by  (4.1)  -  (4.4),  or  equivalently  by 
the  modified  dynamics  (4.6)  with  A  —  B  =  0,  serves  as  another  check  of  the  mean  so¬ 
lution.  Since  the  stochastic  inputs  are  unbiased  and  zero  mean,  it  was  expected  that 
the  hybrid  algorithm  mean  solution  would  be  close  to  the  deterministic  one,  which  is 
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Figure  4.3.  Hybrid  algorithm  and  MCS  variance  estimates  of  x\ (t),  X2 (f),  and  u(t). 

indeed  the  case.  This  check  was  possible  by  virtue  of  the  problem  formulation  and 
may  not  be  available  in  more  general  problems.  These  plots  show  that  the  hybrid 
algorithm  and  MCS  expected  value  results  closely  match  each  other  as  quantified 
with  PD  calculations  using  (4.11).  The  PD  in  expected  values  of  x\ (t),  x2(t),  and 
u[t)  is  0.17%,  0.04%,  and  0.02%,  respectively.  Both  the  hybrid  algorithm  and  MCS 
solutions  also  match  the  deterministic  solution  as  expected,  and  the  desired  terminal 
conditions  X\(tf)  and  x2{tf)  are  satisfied. 

Figure  4.3  shows  the  variances  of  the  hybrid  algorithm  solutions  of  Xi(t),  x2 (t), 
and  u(t),  respectively,  using  (3.4).  MCS  results  are  also  included  for  comparison. 
These  plots  show  close  agreement  between  the  hybrid  algorithm  and  MCS  variance 
approximations  with  PD  values  of  0.17%,  1.236%,  and  3.015%  in  variances  of  x\ (t), 
x2(t),  and  u(t),  respectively.  Desired  terminal  conditions,  X\{tf )  and  x2(tf),  are  also 
satisfied  indicated  by  zero  variances  at  the  terminal  time. 
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(b)  Covariance  between  x2{t)  and  u(t) 


Figure  4.4.  Hybrid  algorithm  and  MCS  covariance  estimates  between  X\ (t)  and  u(t), 
X2  (t)  and  u(t),  and  Xi(t)  and  X2  (t). 


Figure  4.4  shows  the  covariances  between  X\(t)  and  u(t),  x2(t)  and  u(t),  and  X\ (t) 
and  x2(t),  respectively,  calculated  using  (3.5).  Once  again,  MCS  results  are  included 
for  comparison.  These  plots  further  characterize  the  statistics  of  the  solution  by  show¬ 
ing  correlations  between  X\{t)  and  u(t),  x2(t)  and  u(t),  and  X\{t)  and  x2(t).  Since  the 
dynamics  in  (4.6)  are  nonlinear  and  coupled  in  the  state  variables,  it  was  expected 
that  the  output  distributions  would  be  correlated,  as  indicated  by  the  non-zero  co- 
variances.  There  are  more  noticeable  differences  in  these  results  where  the  maximum 
PD  in  covariances  between  x±(t)  and  u(t)  and  x2{t)  and  u(t)  is  approximately  14% 
and  4.3%  in  the  covariance  between  x\ (t)  and  x2  (t).  Even  with  more  pronounced 
differences  in  the  covariance  estimates  between  the  two  methods,  both  sets  of  results 
follow  the  same  general  trend. 

Closer  agreement  between  the  hybrid  algorithm  and  MCS  expected  value,  vari- 
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ance,  and  covariance  estimates  can  be  obtained  by  increasing  the  fidelity  of  the  MCS. 
Decreasing  the  convergence  tolerance  by  an  order  of  magnitude,  to  5  x  10”6,  yields 
between  28%  and  67%  improvement  as  can  be  seen  in  Table  4.1.  The  goal  of  running 
Table  4.1.  Difference  between  hybrid  algorithm  and  MCS  results 


16,500  Sample  MCSa 

74,500K  Sample  MCSb 

s 

Max  Difference 

PD 

Max  Difference 

PD 

Change 

(\SmCS  ~  Salg\ ) 

(%) 

(\SmCS  ~  Salg\ ) 

(%) 

(%) 

Mean 

Xl  (t) 

1.30  x  10”3 

0.1703 

7.88  x  lO'4 

0.1058 

-37.87 

x2{t) 

1.50  x  10”3 

0.0415 

9.45  x  10~4 

0.0224 

-45.93 

u{t) 

6.60  x  10”3 

0.02355 

4.67  x  10~3 

0.0169 

-28.36 

Variance 

Xl  it) 

3.20  x  10"5 

0.1742 

4.40  x  10”5 

0.2396 

37.54 

x2(t) 

2.16  x  10"4 

1.236 

1.05  x  10~4 

0.4021 

-67.47 

u(t) 

8.80  x  10”3 

3.015 

3.32  x  10~4 

1.142 

-62.12 

Covariance 

Xi  (t),u(t) 

3.0  x  10"3 

13.56 

1.03  x  10~3 

5.039 

-62.84 

x2(t),u(t) 

3.6  x  10"3 

14.32 

1.19  x  10“3 

5.203 

-63.67 

Xl 

7.74  x  10“4 

4.321 

4.470  x  10"4 

2.396 

-44.55 

a  MCS  tolerance  =  5  x  10  5 

,  16.5 K  samples,  137.8  minutes  processing 

b  MCS  tolerance  =  5  x  10  6,  74. 5A'  samples,  681.7  minutes  processing 


the  second  MCS  was  to  gain  insight  into  the  cause  of  noticeable  PD  between  the  two 
methods,  particularly  in  the  covariance  approximations.  If  the  PD  values  did  not 
change  appreciably  by  increasing  fidelity  of  the  MCS,  then  it  would  be  reasonable 
to  conclude  the  parameters  of  the  hybrid  algorithm,  such  as  number  of  collocation 
points  and  order  of  polynomial  basis  sets,  would  need  to  be  adjusted  to  improve  the 
quality  of  the  hybrid  algorithm  solution.  Conversely,  if  the  correlation  improved,  then 
it  could  be  concluded  that  the  MCS  convergence  parameters  could  be  better  selected 
to  provide  better  comparison.  Table  4.1  shows  that  decreasing  the  MCS  convergence 
tolerances  by  an  order  of  magnitude  significantly  improved  correlation  with  the  hybrid 
algorithm,  with  most  noticeable  improvement  in  the  covariance  approximations,  but 
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Figure  4.5.  Hybrid  algorithm  cost  approximation  as  function  of  uncertain  inputs  A  and 
B. 


more  than  quadrupled  the  number  of  random  solutions  required  and  corresponding 
processing  time  on  an  Apple  MacBook  Pro  with  a  2.2  gigahertz  quad-core  proces¬ 
sor  and  8  gigabytes  of  memory.  Therefore,  it’s  apparent  that  the  hybrid  algorithm’s 
polynomial  approximation  using  441  collocation  points,  fifth-order  one-dimensional 
polynomial  bases,  and  6.8  minutes  of  computation  time  yields  results  that  are  at  least 
equivalent  to,  and  probably  more  accurate  than,  MCS  solving  74,  500  sampled  OC 
problems. 

Lastly,  Figure  4.5  shows  how  the  cost  varies  as  a  function  of  the  random  inputs. 
This  plot  was  generated  using  a  tensor  product  set  of  test  points  A  and  B  ranging  from 
—3 a  to  3 a  as  inputs  to  the  output  approximation  (3.2)  of  the  cost.  The  surface  plot 
of  the  gPC  cost  values  represents  a  probability-like  distribution  function.  The  lowest 
cost  is  obtained  when  both  A  and  B  are  less  than  zero.  Conversely,  the  largest  costs 
result  when  the  random  inputs  are  both  greater  than  zero.  The  surface  shows  how 
the  costs  vary  over  the  entire  range.  The  star  in  the  figure  shows  the  hybrid  algorithm 
estimation  of  the  expected  cost,  which  is  also  approximately  equal  to  the  deterministic 
cost.  This  plot  is  effective  in  visualizing  the  effects  the  uncertain  parameters  have 
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on  the  OC  solution.  Since  the  cost  is  a  functional  of  the  states  and  control,  thus 
implicitly  a  functional  of  the  uncertain  parameters,  increasing  or  decreasing  cost 
resulting  from  realizations  of  the  random  parameters  gives  an  indication  of  overall 
increase  or  decrease  in  magnitudes  of  the  state  and  control  vectors. 

4.3  Summary 

In  this  chapter,  a  nonlinear  OC  problem  was  considered  where  Gaussian  random 
parameters  were  introduced  into  the  state  equations.  Following  the  gPC  descrip¬ 
tion  in  section  2.4.2,  the  problem  was  formulated  in  a  form  suitable  for  application 
of  the  hybrid  GPM-gPC  algorithm  presented  in  Chapter  III.  The  output  of  the 
algorithm  is  a  set  of  polynomial  approximation  functions  for  x\ (f;  A,  B),  X2(t]  A,  B), 
u(t ;  A,  B ),  and  A,  S),x2(t;  A,  B),u(t ;  A,  B ))  whose  expected  values,  variances, 

and  covariances,  are  found  using  the  expansion  coefficients  and  describe  the  statistical 
properties  of  the  solution.  Comparing  the  statistical  properties  determined  by  appli¬ 
cation  of  the  hybrid  algorithm  with  the  MCS  results  demonstrated  that  the  hybrid 
algorithm  effectively  quantifies  the  effects  of  uncertainty  with  comparable  accuracy 
while  requiring  dramatically  fewer  sample  points  and  associated  GPOPS  determinis¬ 
tic  solutions  and  computation  time.  The  next  chapter  will  apply  the  hybrid  algorithm 
to  another  type  of  optimization  problem,  a  trajectory  optimization  problem,  that  is 
representative  of  a  real-world  mission  planning  scenario,  a  target  application  for  this 
research. 
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V.  Trajectory  Optimization  Problem 


A  trajectory  optimization  problem,  notionally  sketched  in  Figure  5.1,  is  consid¬ 
ered  to  demonstrate  the  ability  of  the  hybrid  GPM-gPC  algorithm  to  quantify 
the  effects  of  uncertain  parameters  on  an  optimal  trajectory  solution.  The  scenario 
was  designed  to  represent  an  aircraft  traveling  to  a  desired  target  location  through 
an  environment  where  there  are  potential  risks  of  lethal  engagements.  The  objective 
was  to  find  the  optimal  path  in  a  two-dimensional  space  that  takes  a  vehicle  from  an 
initial  position  to  a  target  location  while  minimizing  the  probability  that  it  will  be 
killed  by  the  threats  whose  locations  are  uncertain.  It’s  assumed  that  at  some  point 
in  time  the  center  locations  were  perfectly  known  but  have  possibly  moved  in  the 
time  between  intelligence  gathering  and  mission  planning  according  to  an  assumed 
probability  distribution.  The  deterministic  TO  problem  will  be  described  first,  which 
will  form  the  basis  for  the  subsequently  discussed  stochastic  problem. 

5.1  Problem  Formulation 

5.1.1  Deterministic  Baseline  Problem. 

To  begin  adding  details  to  the  notional  scenario  in  Figure  5.1,  a  two-dimensional 
grid  of  50  x  50  nautical  miles  (NM)  was  chosen.  Three  threats  with  effective  ring 
diameters  of  approximately  20  NM  were  arbitrarily  placed  in  the  grid  at  (10  NM,  30 
NM),  (25  NM,  15  NM),  and  (35  NM,  30  NM).  The  target  was  chosen  to  be  located  at 
(25  NM,  50  NM).  Threat  and  target  locations  can  be  easily  changed  to  accommodate 
user-supplied  scenario  specifications.  Likewise,  the  diameters  of  the  threat  rings  can 
be  adjusted  and  are  not  constrained  to  be  uniformly  sized.  Units  of  NM  and  knots 
(kts)  were  used  since  they  are  familiar  to  aircrews  and  mission  planners  who  have 
interest  in  this  type  of  problem. 
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Figure  5.1.  Notional  sketch  of  the  trajectory  optimization  problem 

The  vehicle  dynamics  are  generically  represented  with  the  Dubins  model  in  (5.1). 

xi(t)  =  V  cos (0(t)) 

x2(t)  =  V  sin(0(£))  (5.1) 

9(t)  =  u(t ) 

This  model  was  used  to  avoid  adding  the  complexity  of  developing  and  coding  a 
model  for  a  specific  vehicle’s  dynamics,  which  is  not  necessary  to  demonstrate  the 
hybrid  algorithm’s  performance.  The  state  and  control  vectors  are  defined  as  x(f)  = 
[xi (t),  x2(t),  9{t)]T  G  M3  and  u(t)  =  [0(f)]  G  M1,  where  x\  and  x2  describe  the 
vehicle’s  position  in  the  two-dimensional  grid  space  and  6  is  the  heading  angle.  The 
vehicle  speed  (V)  is  assumed  to  be  a  constant  470  kts,  chosen  based  on  a  mission 
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profile  of  a  generic  large  aircraft,  and  the  control  is  bounded  as: 


,  Nl  Id 
u(t)\  <  —— 


•m%n 


(5.2) 


where  the  minimum  turn  radius  ( Rmin )  based  on  the  constant  speed  is  estimated  to 
be  11,326  feet.  The  initial  and  terminal  boundary  conditions  are: 


x(t0)=  x(-l)  =  [0,0,1.1071]t 
x<Xf)  =  x(!)  =  [25,  50,  free]T 


(5.3) 


where  tf  is  unspecified,  or  free.  Lastly,  admissible  paths  are  restricted  to  lie  within 
the  boundaries  of  the  grid,  stated  in  the  form  of  (2.72)  as: 


[-ZiM.  fciM -50),  -x2(t),  (x2(t)  -  50)]t  <  0 


(5.4) 


These  boundaries  were  chosen  to  restrict  the  vehicle’s  allowable  area  of  operations, 
thereby  creating  a  scenario  where  the  vehicle  must  navigate  through  the  threat  held 
rather  than  completely  avoid  it. 

The  most  important  element  in  setting  up  the  TO  problem  is  to  appropriately 
choose  a  cost  functional  in  the  form  of  (2.69)  that  leads  to  the  trajectory  that  allows 
the  aircraft  to  reach  the  target  while  minimizing  the  vehicle’s  exposure  to  lethal 
threats  and  not  prohibiting  it  from  passing  through  higher  threat  areas  of  the  space. 
The  integrand  of  the  Lagrange  term  in  (2.69)  must  be  chosen  to  define  the  areas 
of  the  space  shown  in  Figure  5.1  that  pose  the  greatest  risk  to  the  vehicle.  In  an 
obstacle  avoidance  problem,  the  threat  shapes  would  be  treated  as  path  constraints, 
as  investigated  by  Gong  et  al.  [44]  and  Lewis  et  al.  [59],  making  any  trajectory  that 
passes  through  those  areas  of  the  space  inadmissible.  Numerical  solvers  would  only 
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return  solutions  that  totally  avoid  those  areas  of  the  space  or  output  a  message 
stating  that  no  optimal  solution  exists  that  satisfies  these  constraints.  However, 
in  this  case,  the  threats  are  treated  as  regions  of  the  space  that  represent  varying 
probabilities  that  the  vehicle  will  be  killed,  not  as  hard  obstacles  that  should  be  totally 
avoided.  The  threat  rings  represent  likelihood,  or  probabilities,  that  traveling  into 
those  areas  would  result  in  the  vehicle  being  eliminated.  Random  elements,  discussed 
later  in  this  section,  are  introduced  through  uncertainty  of  the  exact  center  locations 
of  the  threat  rings,  making  it  possible  for  the  threat  rings  to  overlap  and  impossible 
to  completely  avoid  the  threats.  Therefore,  the  integrand  of  the  cost  functional, 
g(x(r),  u(t),  r;  to,  t/),  should  be  something  that  describes  the  probabilities  of  kill 
(POK)  in  the  space,  assuming  that  the  highest  probabilities  will  be  near  the  center  of 
the  threat  rings  and  decrease  with  distance  away  from  the  center.  Minimizing  such 
a  functional  will  allow  the  vehicle  to  pass  through  regions  of  the  space  occupied  by 
the  threat  rings  while  finding  the  least  threatening  path  to  the  target.  A  bi- variate 
Gaussian  PDF  for  each  threat  was  used  to  describe  the  high-threat  regions  of  the  two 
dimensional  space,  written  as: 


9A-)  = 


2,7T(Jixl  CTj 


-exp 


Oi(r)  -  p<xif  (x2(r)  -  X2 f 

o  l  o 


c 7 ; 


l,X\ 


a, 


Z,X2 


(5.5) 


where  the  mean  values,  and  Hi,X2,  and  the  standard  deviations,  oy,X]  and  oyX2, 
define  the  center  and  variation  in  lethality  of  the  ith  threat  in  the  x\  and  X2  directions 
for  i  —  1, . . . ,  3.  Figure  5.2  depicts  the  high  POK  regions  using  the  previously  stated 
center  locations  and  <Tj>xl  =  oyX2  =  5  NM  to  create  rings  with  20  NM  effective 
diameters. 

One  final  consideration  is  necessary  in  defining  the  cost  functional.  Looking  at 
Figure  5.2,  it  is  obvious  that  multiple  trajectories  may  exist  that  have  equal  minimum 
POK,  meaning  that  a  unique  solution  may  not  exist,  resulting  in  GPOPS  failure  to 
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return  a  solution.  To  minimize  this  possibility,  the  Mayer  term  in  the  cost  functional 
should  be  chosen  to  drive  the  software  to  return  the  minimum  POK  path  with  shortest 
travel  time  to  the  target.  Including  the  Mayer  term  in  the  cost  functional  gives: 


(5.6) 


where  the  weighting  factor,  W,  on  tj  is  used  to  make  sure  the  Mayer  and  Lagrange 
terms  are  on  the  same  order  of  magnitude  to  avoid  having  the  minimum  time  require¬ 
ment  dominate  the  solution.  For  this  problem,  W  =  1CT5  was  used. 

Finally,  the  TO  problem  can  be  stated  as:  Minimize  J  (5.6)  subject  to  the  dy¬ 
namics  (5.1),  control  constraint  (5.2),  boundary  conditions  (5.3),  and  path  constraints 
(5.4).  This  is  a  continuous-time  Bolza-type  TO  problem  that  is  nonlinear  in  both  the 
cost  functional  and  the  state  dynamics,  with  fixed  final  state  and  free  final  time, 
and  constrained  boundary  conditions  and  control  input.  The  solution  to  the  problem 
described  thus  far  will  be  referred  to  as  the  deterministic  solution,  i.e.  the  noise-free 
solution,  which  was  found  using  GPOPS  and  is  shown  in  Figure  5.2.  The  mean  solu¬ 
tion  of  the  stochastic  variant  discussed  in  the  next  section  will  be  compared  to  this 
solution  and  should  closely  match  it  since  the  uncertainties  will  be  assumed  to  be 
zero  mean. 

5.1.2  Stochastic  Problem. 

The  stochastic  problem  is  formulated  in  this  section  by  modifying  the  deterministic 
version  in  the  previous  section  and  then  following  the  development  in  section  2.4.2. 
Uncertainty  is  added  to  the  deterministic  problem  by  introducing  three  random  input 
parameters,  A1,  A%,  and  Ag,  and  making  the  following  substitutions  indicated  by  the 
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Figure  5.2.  Threat  ring  locations  and  deterministic  solution  to  the  trajectory  optimiza¬ 
tion  problem 


(— y)  into  the  Lagrange  term,  (5.5),  of  the  cost  functional,  (5.6). 


fJ'ijXi 

^  l^i,x i  “1“  ^z 

f^i,X2 

^  \^i,X2  ^z 

The  integrand  of  the  modified  cost  functional  becomes: 


#(•)  = 


‘2i7T(JiiXi&i,X2 


exp 


(xi(t)  -  (nijXl  +  At ))2  {x2{t)  -  (pi,x 2  +  Ai )) 


+ 


<7 


hxi 


<7 


i,x2 


resulting  in  the  modified  cost  functional: 


J(.)  =  mf  + 


tf  -  h 
2 


9t(%i Ah  t0,t/)dT 


(5.7) 


(5.8) 


where  the  Tlj’s  are  assumed  to  be  independent  RVs  whose  distributions  will  be  dis¬ 
cussed  later  in  this  section.  The  random  inputs  thus  defined  and  written  into  (5.7) 
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and  (5.8)  introduce  uncertainties  on  center  locations  of  the  threats,  transforming  J, 
x,  u,  and  tf  into  RVs  whose  solutions  will  have  statistical  properties. 

As  in  Chapter  IV,  the  cost  functional  is  not  written  with  an  expectation  operator 
since  quantification  of  the  effects  of  uncertain  parameters  on  J(x),  x(i),  u(t),  and  tf 
solutions  is  sought  and  there  is  no  way  to  control  the  uncertainties  or  minimize  their 
effects.  So,  the  cost  functional  as  expressed  by  (5.7)  and  (5.8)  is  used  when  solving 
each  of  the  Q  sampled  deterministic  problems,  and  the  set  of  solutions  is  used  to 
construct  gPC  approximations  of  the  states,  control,  cost,  and  final  time  as  functions 
of  the  random  inputs  to  assess  the  effects  of  those  uncertainties.  Additionally,  the 
dynamics  (5.1),  control  bounds  (5.2),  boundary  conditions  (5.3),  and  path  constraints 
(5.4)  remain  the  same  with  the  addition  of  uncertainty. 

Following  the  development  in  section  2.4.2,  the  following  definitions  are  made  to 
identify  the  key  details  needed  to  apply  the  hybrid  algorithm: 

•  State  variables:  x  =  (xi,x2 ,  $)  G  M3  (J  =  n  =  3) 

•  Stochastic  input  parameters  (random  variables):  p  =  (A1:  A2,  A3 )  G  M3 

(N  =  3). 

•  Outputs  or  observables:  z  =  (xi,x2,9,u,tf,  J )  G  R6  (K  =  6) 

The  PDFs  of  Au  A2,  and  A3  are  denoted  by  pi(A^),  p2(A2),  and  p3(A3),  respec¬ 
tively.  The  joint  PDF  is  given  by  (2.106)  as: 

3 

P (p)  =  \\pi{Pi)  =  Pr{Ai)p2(A2)pz{A3)  (5.9) 

i= 1 

In  this  work,  four  combinations  of  the  PDFs  of  A1,  A2,  and  A3  are  considered  as 
shown  in  Table  5.1,  noting  that  the  notation  for  Gaussian  (normal)  distributions  is 
A/"(/i,  a),  and  for  uniform  distributions  is  U(a,  b ). 
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Table  5.1.  Combinations  of  PDFs  pi(Ai ),  p2(d,g),  and  ,03(43)  and  resulting  finite  sample 
space,  r 


Combination 

P2(Ag) 

^3(^3) 

r 

1 

W(0,0.5) 

A/”(0, 0.5) 

A/”(0, 0.5) 

[-2,2]  x  [-2,2]  x  [-2,2] 

2 

W(-l,l) 

U{- 1,1) 

W(-l,l) 

[-1,1]  x  [-1,1]  x  [-1,1] 

3 

A/”(0, 0.5) 

U{- 1,1) 

A7(0,0.5) 

[-2,2]  x  [-1,1]  x  [-2,2] 

4 

W(-l,l) 

Af  (0,0.5) 

W(-l,l) 

[-1,1]  x  [-2,2]  x  [-1,1] 

Hermite  polynomial  bases  collocated  at  Gauss-Hermite  quadrature  points  were 
used  for  gPC  expansions  associated  with  Gaussian  distributions  and  Legendre  polyno¬ 
mial  bases  collocated  at  Gauss-Legendre  points  were  used  for  gPC  expansions  associ¬ 
ated  with  uniform  distributions  as  suggested  in  [87]  and  [88].  As  discussed  in  Chapter 
IV,  the  statistical  version  of  the  Hermite  polynomials  was  used  where  Gauss-Hermite 
quadrature  points  were  scaled  by  \j2a2  and  shifted  by  /1  and  the  quadrature  weights 
scaled  by  (y7^)-1.  Additionally,  the  Gauss-Legendre  quadrature  weights  were  scaled 
such  that  the  weights  in  the  set  sum  to  one  so  that  the  Legendre  polynomials  ap¬ 
propriately  approximate  uniform  probability  distributions.  Recall  there  are  q  points 
in  each  random  dimension,  which  are  calculated  by  finding  the  roots  of  either  the 
qth  Hermite  polynomial,  or  the  qth  Legendre  polynomial,  Lq ,  as  dictated  by  the 
assumed  PDF  of  the  uncertain  parameter,  and  scaled  accordingly.  The  number  of 
collocation  points  in  the  three  random  dimensions,  q±,  g2,  and  q 3,  was  chosen  to  be 
7  to  provide  a  balance  between  computation  time  and  accuracy  of  the  solution  when 
compared  to  MCS,  resulting  in  Q  =  343  collocation  nodes  with  the  finite  probability 
domains  shown  in  Figure  5.3  and  listed  in  Table  5.1. 

The  one- dimensional  polynomial  spaces  were  chosen,  as  given  in  (2.108),  to  be 
defined  by  the  following  Hermite  ( H )  and  Legendre  ( L )  polynomial  basis  sets  as 
shown  in  Table  5.2  for  the  four  cases  considered.  In  Table  5.2,  the  subscripts  on  H 
and  L  indicate  the  order  of  the  polynomial  basis  elements.  Thus,  the  3-dimensional 
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(c)  Configuration  #3  (d)  Configuration  #4 


Figure  5.3.  Collocation  points:  Tensor  grid  of  343  points  (7  in  each  dimension)  used  as 
random  inputs  Ai,  Ag,  and  A$. 

polynomial  space,  defined  using  the  tensor  product  in  (2.110),  is: 

Wn  =  {«!=„  <g){02}^_o  ®  {«*,o  (5.10) 

Selecting  the  entire  set  of  basis  functions,  Pc  =  21  in  (2.111),  yields  512  total  basis 
elements,  where  each  one-dimensional  basis  set  is  at  most  seventh  order  and  the 
highest  order  of  the  tensor  product  set  is  21.  The  number  of  basis  elements  was  chosen, 
all  of  which  are  used  in  computations,  to  provide  a  balance  between  computational 
burden  and  accuracy  of  the  solution  when  compared  to  MCS. 

This  section  has  presented  all  of  the  details  necessary  to  implement  the  hybrid 
GPM-gPC  algorithm  steps  described  in  Chapter  III.  The  results  of  applying  the 
algorithm  to  the  sample  problem  considered  in  this  research  are  presented  in  the  next 
section. 


106 


Table  5.2.  One-Dimensional  polynomial  basis  sets 


Combination  Polynomial  Basis  Set 

01  02  03 


1 

{H0,. 

■  ■,h7} 

{Ho,  ■ 

■  ,h7} 

{Ho,. 

■  ■,h7} 

2 

{Lo,. 

■  ■ ,  l7} 

{Lo,  ■  ■ 

■  ■ ,  l7} 

{Lo,  ■ 

■  ■ ,  l7} 

3 

{Ho,. 

..,h7} 

{£o, . , 

■  ■ ,  l7} 

{Ho,- 

..,h7} 

4 

{L0,. 

■  ■ ,  l7} 

{#0,  • 

■  ,h7} 

{Lo,  ■ 

■  ■ ,  l7} 

5.2  Results 

Results  of  applying  the  hybrid  GPM-gPC  algorithm  to  the  TO  concept  demonstra¬ 
tion  problem  are  presented  in  this  section.  The  expected  value  solutions  and  variances 
are  plotted  for  x\ (r),  x2(r),  0(r),  and  u(r)  and  tabulated  for  J  and  tf  for  each  of  the 
configurations  listed  in  Table  5.1.  Covariances  between  the  states  xi(r)  and  x2(r), 
xi(t)  and  0(r),  and  x2(r)  and  0(r),  as  well  as  between  the  states  and  control,  x\  (r) 
and  u(t),  x2(t)  and  w(t),  and  0{r)  and  w(t),  are  also  shown.  The  expected  value, 
variance,  and  covariance  results  determined  by  the  hybrid  algorithm  are  compared  to 
the  MCS  results  for  verification.  Lastly,  the  hybrid  algorithm  solution  statistics  are 
used  to  estimate  the  probabilities  that  the  vehicle  will  be  killed  during  the  course  of 
the  mission.  Since  this  is  a  free  final  time  problem,  expected  values,  variances,  and 
covariances  are  plotted  versus  the  time  vector  r.  Stochastic  computations  evaluating 
the  hybrid  algorithm  output  function  (3.2),  expansion  coefficients  (3.1),  and  statis¬ 
tical  quantities  derived  from  the  expansion  coefficients  are  not  possible  in  the  time 
domain  since  the  time  points  in  the  Q  sampled  solutions  are  not  the  same  due  to  the 
GPOPS  transformation  from  the  r  domain  to  the  time  domain  using  (2.73).  The  r 
vectors,  however,  remain  consistent  throughout  regardless  of  the  differing  terminal 
times  and  allows  for  stochastic  computations  using  a  consistent  time  reference.  In  a 
fixed  final  time  problem,  these  stochastic  computations  can  be  performed  using  the 
time  domain. 
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As  in  Chapter  IV,  the  MCS  was  accomplished  by  generating  random  samples  of 
the  stochastic  variables,  (Al7  A%,  A3),  taken  from  either  Gaussian  or  uniform  distri¬ 
butions  as  appropriate  in  accordance  with  the  four  cases  specified  in  Table  5.1.  Each 
sample  was  subsequently  inserted  into  the  Lagrange  term  (5.7)  of  the  stochastic  cost 
functional  (5.8),  creating  a  set  of  deterministic  TO  problems  that  were  solved  using 
GPOPS.  The  ensemble  of  solutions  was  used  to  calculate  the  MCS  estimations  of 
expected  values  (means),  variances,  and  covariances.  Convergence  was  checked  after 
every  500  samples  and  was  achieved  when  the  maximum  change  in  expected  values 
of  the  state  trajectories  from  the  previous  interval  was  below  an  arbitrarily  chosen 
tolerance  of  1  x  10~4.  Expected  value,  variance,  and  covariance  properties  of  the  hy¬ 
brid  algorithm  solutions  are  compared  to  the  MCS  results  using  the  the  PD  equation 
(4.11)  introduced  in  Chapter  IV  and  are  presented  in  Table  5.3  discussed  later  in  this 
section. 

Expected  value,  or  mean,  approximations  indicate  the  most  likely  solution  to  the 
TO  problem.  Figures  5.5  -  5.8  show  the  mean  solutions  of  aq (r),  x2(r),  0(r),  and 
u(t)  for  each  of  the  four  cases  listed  in  Table  5.1.  The  three  curves  on  each  of 
these  plots  show  the  hybrid  algorithm  expected  value  solutions,  the  MCS  expected 
value  solutions,  and  the  deterministic  solution.  The  hybrid  algorithm  expected  value 
approximation  for  each  state  and  control  variable  is  given  by  the  first  coefficient 
in  each  expansion  as  stated  in  (3.3).  Bars  indicating  two  standard  deviations  are 
included  to  quantify  how  much  the  solution  varies  from  the  mean.  The  solution 
to  the  deterministic  TO  problem  defined  by  (5.1)  -  (5.6),  or  equivalently  by  (5.1)  - 
(5.4)  and  the  stochastic  cost  functional  (5.8)  with  A*  =  0,  serves  as  another  check 
of  the  mean  solution.  Since  the  Gaussian  stochastic  inputs  are  unbiased  and  zero 
mean  and  the  uniform  stochastic  inputs  are  symmetric  about  zero,  it  was  expected 
that  the  hybrid  algorithm  mean  solutions  would  be  close  to  the  deterministic  ones  as 
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shown  in  the  expected  value  figures.  This  check  was  possible  by  virtue  of  the  problem 
formulation  and  may  not  be  available  in  more  general  problems.  These  plots  show 
that  the  hybrid  algorithm  and  MCS  expected  value  results  closely  match  each  other 
as  quantified  with  PD  calculations  using  (4.11)  and  shown  in  Table  5.3.  The  PDs  in 
expected  values  of  aq(r),  £2 (r),  9(r),  and  u(r)  are  generally  on  the  order  of  10”2  and 
at  most  around  one  percent  in  the  u(r)  approximation  in  cases  #3  and  #4.  Both 
the  hybrid  algorithm  and  the  MCS  solutions  also  match  the  deterministic  solution  as 
expected,  and  the  desired  terminal  conditions  x\ (tf)  and  X2  (tf)  are  satisfied. 

Variance  approximations  to  the  stochastic  problem  indicate  how  much  the  solu¬ 
tions  to  the  TO  problem  could  deviate  from  the  most  likely  solutions  depending  on 
the  values  that  the  uncertain  parameters  can  assume.  Figures  5.9  -  5.12  show  the 
variances  of  the  hybrid  algorithm  solutions  of  x\ (r),  x2(r),  9(t),  and  u(t)  for  each 
of  the  four  cases  listed  in  Table  5.1,  calculated  using  (3.4).  MCS  results  are  again 
included  for  comparison.  These  plots  show  close  agreement  between  the  hybrid  al¬ 
gorithm  and  MCS  variance  approximations  of  Xi(r),  £2(t),  9(t),  and  u(r)  with  PD 
values,  as  shown  in  Table  5.3,  ranging  from  0.901%  to  4.55%  and  predominantly  in 
the  one  to  two  percent  range.  Desired  terminal  conditions,  X\ (tf)  and  x2(tf )  are  also 
satisfied  as  indicated  by  zero  variances  at  the  terminal  time. 

Covariance  estimations  further  characterize  the  distributions  of  the  stochastic 
state  and  control  solutions  by  providing  indications  of  correlations  between  x\  (r) 
and  x2(r),  x\ (r)  and  9(r),  x2(r)  and  9(t),  x\ (r)  and  u(r),  x2(r)  and  u(r),  and  6(r) 
and  u(t).  Figures  5.13  -  5.16  show  these  covariance  estimates  approximated  by  the 
hybrid  algorithm,  calculated  using  (3.5),  and  compares  them  with  MCS  estimates 
for  each  of  the  four  cases  listed  in  Table  5.1.  The  covariance  plots  show  that  the 
state  variables  and  control  outputs  are  dependent  even  though  the  random  inputs 
are  assumed  to  be  independent,  which  is  as  expected  since  the  state  trajectories  and 
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uncertain  parameters  are  related  through  the  nonlinear  cost  functional  (5.8).  As  with 
the  expected  value  and  variance  estimates,  these  figures  show  agreement  between  the 
hybrid  algorithm  and  the  MCS  results,  which  are  confirmed  quantitatively  in  Table 
5.3,  where  the  PD  values  are  between  0.7  and  4.6  percent.  Note  that  the  table  also 
shows  two  suspiciously  high  PD  comparisons:  23.92%  in  the  covariance  between  X\{r) 
and  9{t)  in  case  #1  (Table  5.3(a))  and  54.16%  in  the  covariance  between  9{r)  and 
u[r)  in  case  #4  (Table  5.3(d)).  However,  these  apparently  large  differences  between 
hybrid  algorithm  and  MCS  approximations  are  not  evident  in  Figures  5.13(b)  and 
5.16(f).  These  exceptions  are  due  to  limitations  of  using  (4.11)  in  cases  where  the 
difference  between  the  quantities  being  compared  (numerator)  is  very  small  but  the 
values  themselves  are  far  enough  apart  such  that  the  average  (denominator)  results 
in  an  unexpectedly  high  PD,  even  though  significant  differences  are  not  obvious  on 
the  graphs.  Therefore,  it  was  concluded  that  these  anomalies  do  not  demonstrate  ap¬ 
preciable  disagreement  between  the  hybrid  algorithm  and  MCS  and  are  a  limitation 
of  the  PD  metric. 

Expected  value  and  variance  results  for  estimating  the  cost,  J,  and  final  time,  tf, 
are  given  in  Table  5.4  and  Table  5.5,  respectively.  The  hybrid  algorithm  and  MCS 
approximations  for  the  expected  values  and  variances  in  both  variables  closely  match 
each  other  as  shown  by  the  small  PDs  given  in  the  tables  and  are  actually  equal,  to 
four  decimal  places,  in  expected  value  of  the  final  time.  Furthermore,  the  expected 
values  of  both  J  and  tf  agree  with  the  deterministic  solution. 

The  expected  value  plots  (Figures  5.5  -  5.8),  along  with  the  variance  plots  (Figures 
5.9  -  5.12)  and  covariance  plots  (Figures  5.13  -  5.16)  give  the  most  likely  solution 
to  the  stochastic  trajectory  optimization  problem  and  describe  the  nature  of  the 
output  distributions  by  quantifying  the  affects  of  the  random  parameters  on  the 
output  solutions.  Combined  with  the  information  in  Table  5.3,  these  results  show 
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that  the  hybrid  algorithm  provides  solutions  to  the  stochastic  problems  that  are  at 
least  equivalent  to  the  widely-accepted  MCS  method  while  significantly  reducing  the 
computational  workload.  Having  a  mix  of  probability  distributions  and  thus  a  mix 
of  polynomial  basis  sets  also  does  not  appear  to  have  any  effect  on  the  ability  of  the 
hybrid  algorithm  to  produce  accurate  solution  approximations  that  correlate  well  with 
MCS  approximations.  Notes  included  in  Table  5.3  show  that  the  hybrid  algorithm 
is  able  to  produce  equivalent  approximations  using  seventh-order  one-dimensional 
polynomial  bases  and  343  sampled  deterministic  solutions  requiring  five  to  six  minutes 
of  processing  on  an  Apple  MacBook  Pro  computer  with  a  2.2  gigahertz  quad-core 
processor  and  8  gigabytes  of  memory,  whereas  the  MCS  required  between  17.5K  and 
34K  sampled  deterministic  solutions  and  up  to  6.5  hours  of  processing. 

The  most  valuable  product  to  mission  planners  and  aircrews  is  not  the  set  of 
statistics  describing  expected  values,  variances,  and  covariances  and  how  well  the 
hybrid  algorithm  results  compare  with  the  MCS  results,  but  rather  an  estimation  of 
the  probability  that  they  will  suffer  from  lethal  engagements  with  the  threats  while 
executing  the  mission.  The  discussion  that  follows  leads  to  an  estimate  of  the  POK 
for  the  mission  scenario  by  re-packaging  the  aforementioned  results  for  case  #1  where 
the  uncertainties  related  to  the  center  locations  of  the  three  threats  are  assumed  to 
have  zero  mean  Gaussian  distributions  with  0.5  NM  standard  deviations.  The  analysis 
that  follows  could  be  repeated  for  the  other  three  cases,  but  does  not  add  significantly 
new  or  noteworthy  results  and  is  thus  omitted. 

To  do  the  evaluation,  it  was  assumed  no  new  information  is  available  on  the  threat 
locations.  Therefore,  the  best  guess  at  a  mission  plan  is  to  fly  the  mean  trajectory 
shown  in  Figure  5.5,  noting  that  if  updated  information  were  to  become  available, 
the  hybrid  output  approximation  function  (3.2)  could  be  evaluated  to  provide  the 
crews  with  a  new  navigation  solution  with  minimum  POK.  To  assess  minimum  and 
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maximum  expected  POKs,  the  analysis  was  conducted  as  if  the  threats  have  moved 
in  best  and  worst  case  scenarios  while  assuming  the  mean  trajectory  is  followed.  The 
best  case  was  modeled  by  assuming  the  threats  have  moved  away  from  the  mean 
trajectory  by  the  extreme  values  shown  in  Figure  5.3  by  letting  A\  =  —2,  A 2  =  2, 
and  A%  =  2,  resulting  in  the  threat  rings  defined  by  the  POK  PDF,  the  integrand  in 
(5.8),  shown  in  Figure  5.17(a),  effectively  reducing  potential  exposure  to  the  threats 
when  compared  to  the  nominal  threat  configuration.  Conversely,  the  worst  case  was 
modeled  by  assuming  that  the  threats  have  moved  closer  to  the  mean  trajectory  by 
the  extreme  values  by  letting  A\  =  2,  A2  =  —2,  and  A3  =  —2,  resulting  in  the  threat 
rings  shown  in  Figure  5.17(b),  thus  increasing  the  potential  of  fatal  engagements. 
With  these  choices,  the  POK  PDFs  given  in  (5.7)  can  be  written  for  the  best  case  as: 


9i(xi(t),x2(t),t)  = 
g2(x1(T),x2(T),T)  = 
^3(xi(t),X2(t),t)  = 


^exp(~l 
ikexP  (-1 
lkexp  (-1 


Qip)— 8)2  1  02(t)-28)2 

25  ~r  25 

(iri(-r)— 27)2  ,  0k2(t)-17)2 

25  “r  25 

(zi(r)-37)2  32)2 

25  1  25 


(5.11) 


and  for  the  worst  case  as: 


9i(xi(t),x2(t),t)  = 
92(xi(t),x2(t),t)  = 

93(x1(t),x2(t),t)  = 


ltexP  (“I 
lkexP  (“I 
ltexP  ("I 


(xi(r)  — 12)2  (x2(r)— 32)2 


25 

Qi(t)— 23)2 
25 

(xi(t)- 33)2 
25 


+ 

+ 


25 

(x2(t)-13)2 

25 

(x2(t)— 27)2 
25 


(5.12) 


recalling  that  the  full  POK  PDF  is  given  by: 

3 

9(zi(t),x2(t),t)  =y^9i(x1(T),x2(r),T)  (5.13) 

1=1 


The  POK  as  a  function  of  r  can  be  determined  by  integrating  (5.13)  over  some 
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region  of  the  two-dimensional  space  surrounding  the  mean  trajectory  at  each  time 
point.  The  region  of  the  space  was  determined,  based  on  the  expected  value  trajectory 
shown  in  Figure  5.5  and  variance  results  shown  in  Figure  5.9,  by  selecting  the  upper 
and  lower  bound  trajectories,  x„&  and  x«,,  as: 

xub=  (E  [xi (t)]  +  10y/var(xi(r)),E  [x2(r)]  +  lO^ar^r)))  (5.14) 
x/h  =  (e  [xi(r)j  -  10a/ nar(xi(r)),  E  [x2(t)]  -  10y/var(xi(r))  ) 

This  region  is  shown  in  Figure  5.17  and  was  arbitrarily  chosen  to  provide  conservative 
POK  estimates  to  aircrews  and  mission  planners  while  allowing  for  some  flexibility 
in  flight  path.  In  this  figure,  notice  that  the  trajectory  windows  are  similar  for  the 
best  and  worst  case  scenarios  and  very  small  early  in  the  mission  profile  where  there 
is  minimal  threat  to  the  vehicle  necessitating  little  deviation  from  the  minimum  time 
trajectory.  Thus,  the  best  and  worst  case  paths  remain  close  to  the  expected  value. 
As  the  vehicle  moves  into  the  center  of  the  threat  held,  the  threats  pose  greater  risks 
to  the  vehicle  resulting  in  greater  variance  in  the  path  that  should  be  flown  to  mitigate 
the  risks.  These  observations  are  further  evident  in  the  POK  calculation  along  the 
path. 

The  POK  as  a  function  of  r  was  determined  by  constructing  the  Cumulative 
Distribution  Function  (CDF)  of  the  POK  PDF  given  by: 

rx2*i  fxibi  3 ' 

G(ri)=  /  y^Qi{xi(T),x2{T),T)dx1dx2  (5.15) 

^X2,i  xl\,i  2—1 

where  the  counting  index,  i,  refers  to  the  ith  r  point  and  the  limits  of  integration  at 
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each  Ti  point  are: 


xli,i  =  E  [xi(r*)]  _  10>Jvar(xi(Ti)) 


(5.16) 


xfi  =  E  [xi (ri)\  +  10 ^var(xi(Ti)) 


xlli  =  E  [x2(Ti)\  -  10 \Zvar(x2{Ti)) 


x2bi  =  E  [x2(Ti)\  +  10 \/var(x2(ri)) 


The  results  are  shown  in  Figure  5.18.  This  hgure  shows  that  early  in  the  mission,  r  less 
than  -0.6,  the  POK  is  essentially  zero,  then  increases  to  a  maximum  of  approximately 
0.5%  as  the  vehicle  approaches  the  closest  threat,  with  both  scenarios  resulting  in 
similar  threat  levels.  As  the  vehicle  gets  into  the  higher  risk  regions  of  the  space, 
r  greater  than  0.2,  the  best  case  and  worst  case  POK  estimates  diverge  where  the 
best  case  shows  a  maximum  POK  of  approximately  0.25%  and  the  worst  case  peaks 
at  about  1%.  This  information  provides  aircrews  with  estimates  of  how  vulnerable 
they  are  to  the  threats  at  each  point  in  time  as  they  progress  through  the  mission, 
although  they  would  also  be  interested  in  knowing  an  overall  mission  POK  estimate. 

The  mission  POK  was  found  by  integrating  the  POK  CDF  (5.15)  over  the  time 
domain  for  both  best  and  worst  cases,  written  as: 


(5.17) 


The  integration  was  performed  by  applying  Gaussian  quadrature  to  the  data  presented 
in  Figure  5.18  resulting  in  cumulative  POK  estimates  shown  in  Figure  5.19.  The 
anticipated  mission  POKs  are  read  from  the  hgure  at  the  end  time  and  range  from 
0.25%  to  0.5%.  This  information,  combined  with  the  information  contained  in  Figure 
5.18,  tells  planners  and  crews  what  their  expected  overall  mission  POK  is  along  with 
identifying  the  points  during  the  mission  where  they  are  most  vulnerable. 
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xu  NM 


Figure  5.4.  Threat  ring  locations  and  hybrid  algorithm  mean  solution  to  the  modified 
trajectory  optimization  problem  used  in  POK  analysis. 

The  POK  evaluation  was  repeated  for  a  more  challenging  threat  layout  to  further 
demonstrate  the  ability  of  the  hybrid  algorithm  to  produce  POK  estimates  that  are 
meaningful  to  operators  and  planners.  The  locations  of  the  threats  were  modified 
to  yield  greater  interactions  than  was  shown  in  Figure  5.17.  The  modified  scenario 
was  constructed  by  placing  the  threats  at  (10  NM,  27  NM),  (24  NM,  23  NM),  and 
(25  NM,  38  NM)  as  shown  in  Figure  5.4.  The  hybrid  algorithm  was  applied  to  this 
scenario  to  estimate  the  expected  value  trajectory,  also  shown  in  Figure  5.4,  and  the 
variance  information  needed  in  the  POK  analysis.  Comparison  with  an  MCS  is  not 
presented  but  it’s  interesting  to  note  that  the  hybrid  algorithm  generated  solutions 
to  this  modified  scenario  in  26  minutes,  whereas  the  MCS  did  not  converge  after  23 
hours  of  processing  generating  an  ensemble  of  50K  sample  solutions. 

To  begin  the  POK  analysis,  the  best  case  was  once  again  modeled  by  assuming 
the  threats  have  moved  away  from  the  mean  trajectory  by  letting  Ai  =  —2,  A2  =  2, 
and  A%  =  2,  resulting  in  the  threat  rings  shown  in  Figure  5.20(a).  The  worst  case 
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was  also  modeled  as  before  by  assuming  that  the  threats  have  moved  closer  to  the 
mean  trajectory  by  letting  Ai  =  2,  A2  =  —2,  and  A3  =  —2,  resulting  in  the  threat 
rings  shown  in  Figure  5.20(b).  The  modified  POK  PDFs,  given  by  (5.7)  for  the  best 
case  are: 


gi(xi(r),x2(r),T)  = 
g2(x1(r),x2(r),r)  = 
^3(xi(r),x2(r),r)  = 


5 texP  (“I 
5 hexP  (-1 
ikexP  (-5 


Wild s)2 

25 

(j:i(t)-26)2 

25 

Ql(r)-27)2 

25 


+ 

+ 

+ 


(X2(r)— 25)2 
25 

Qe2(t)-25)2' 

25 

(x2(r)-40)2 

25 


(5.18) 


and  for  the  worst  case  are: 


9i(x1(t),x2{t),t)=  5 ^exp 
^2(xi(r),x2(r),r)  =  ^ exp 
fib  (x\  (r)  ,x2(t),t)  —  5 ^exp 


["  Olfr)— 12)2 

+ 

(x2(t)— 29)2 

V  2 

25 

25 

(-! 

'(*i(t)-22)2 

+ 

02(t)-21)2 

V  2 

25 

25 

( - 1 

(xi(r)— 23)2 

+ 

(x2(t)-36)2 

V  2 

25 

25 

(5.19) 


recalling  that  the  full  POK  PDF  is  the  sum  of  the  three  contributions  given  by  (5.13). 

The  POK  as  a  function  of  r  was  determined  by  integrating  (5.13)  over  the  envelope 
surrounding  the  mean  trajectory  defined  by: 

xu6  =  (E  [xi(r)j  +  4-^/ nar(xi(r)),  E  [x2(r)j  +  4yW(xi(r))  )  (5.20) 

X;b  =  (e  [xi(r)]  -  4yW(£i(r)),  E  [x2(r)]  -  4yW(zi(7-))) 

This  region  is  shown  in  Figure  5.20  and  was  arbitrarily  chosen  to  provide  conservative 
POK  estimates  while  allowing  some  flexibility  in  flight  path.  The  results  of  the  spatial 
integration  are  shown  in  Figure  5.21.  This  figure  shows  that  early  in  the  mission 
(r  <  —0.6)  the  POK  is  essentially  zero,  increases  to  a  maximum  of  approximately 
8%  in  the  best  case  and  13%  in  the  worst  case  as  the  vehicle  traverses  the  high  threat 
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region  of  the  space  (—0.6  <  r  <  0.4),  then  becomes  essentially  zero  again  (r  >  0.4) 
for  the  last  part  of  the  mission.  This  information  provides  aircrews  with  estimates  of 
how  vulnerable  they  are  at  each  point  in  time  during  the  mission  and  can  be  used  to 
provide  an  overall  mission  POK  estimate. 

The  mission  POK  estimate  for  the  modified  scenario  was  found  by  integrating  the 
POK  CDF  (5.15)  over  the  time  domain  for  both  best  and  worst  cases  using  Gaussian 
quadrature  applied  to  the  data  presented  in  Figure  5.21.  The  results  are  shown  in 
Figure  5.22,  which  displays  the  mission  POK  as  a  cumulative  total  at  each  time 
point.  The  POK  estimate  covering  the  duration  of  the  mission,  which  is  read  from 
the  figure  at  the  final  time  point,  ranges  between  3.3%  and  6.8%.  The  combination 
of  the  information  shown  in  Figures  5.21  and  5.22  tells  mission  planners  and  aircrews 
their  probability  of  being  killed  based  on  their  position  along  the  planned  trajectory 
and  the  total  probability  of  being  killed  at  each  point  in  time. 

5.3  Summary 

The  concept  demonstration  problem  considered  in  this  chapter  was  designed  to  be 
representative  of  a  real-world  mission  scenario  that  is  of  interest  to  USSTRATCOM. 
It  is  a  nonlinear  TO  problem  designed  to  find  the  path  through  a  two-dimensional 
space  that  minimizes  the  probability  a  vehicle  will  be  killed  by  lethal  threats  whose 
locations  are  uncertain,  to  quantify  the  effects  those  uncertainties  have  on  the  solu¬ 
tion  by  estimating  the  statistical  properties,  and  to  use  the  statistical  properties  to 
estimate  the  probability  of  the  vehicle  being  killed  during  the  mission.  Uncertainties 
in  the  locations  of  the  threats  were  modeled  as  either  Gaussian  or  uniform  random 
variables  that  entered  the  problem  in  the  the  cost  functional.  Following  the  gPC 
description  in  section  2.4.2,  the  problem  was  formulated  in  a  form  suitable  for  ap¬ 
plication  of  the  hybrid  GPM-gPC  algorithm  presented  in  Chapter  III.  The  output 
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of  the  algorithm  provided  a  set  of  polynomial  approximation  functions  for  the  state 
variables,  x(r;  A1:  A2,  _A3),  control,  u(r;  Ai,  A2,  _A3),  cost,  J(A1,  A2,  yf3),  and  terminal 
time,  t/(A1,A2,A3),  whose  expected  values,  variances,  and  covariances,  were  found 
using  the  expansion  coefficients  and  describe  the  statistical  properties  of  the  solu¬ 
tion.  Comparing  the  statistical  properties  determined  by  application  of  the  hybrid 
algorithm  with  the  MCS  results  demonstrated  that  the  hybrid  algorithm  effectively 
quantifies  the  effects  of  uncertainty  with  comparable  accuracy  while  requiring  dra¬ 
matically  fewer  sample  points  and  associated  GPOPS  deterministic  solutions  and 
computation  time.  Probability  of  kill  computations  demonstrated  that  the  hybrid 
algorithm  results  can  be  used  to  assess  risks  associated  with  a  trajectory  solution  in 
a  way  that  is  meaningful  to  mission  planners  and  aircrews. 
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(d)  Expected  Value  of  u[r) 


Figure  5.5.  MCS,  deterministic,  and  hybrid  algorithm  expected  value  estimates  of 
states,  x\ (t),  X2(t),  and  0(r),  and  control,  u(r),  for  configuration  #1. 


Time,  t 

(c)  Expected  Value  of  6{t) 


(d)  Expected  Value  of  u[r) 


Figure  5.6.  MCS,  deterministic,  and  hybrid  algorithm  expected  value  estimates  of 
states,  x\ (r),  X2 (t),  and  9(t),  and  control,  u(t),  for  configuration  #2. 
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(d)  Expected  Value  of  u[r) 


Figure  5.7.  MCS,  deterministic,  and  hybrid  algorithm  expected  value  estimates  of 
states,  x\ (t),  X2(t),  and  0(r),  and  control,  u(r),  for  configuration  #3. 


Time,  t 

(c)  Expected  Value  of  6{t) 


(d)  Expected  Value  of  u[r) 


Figure  5.8.  MCS,  deterministic,  and  hybrid  algorithm  expected  value  estimates  of 
states,  x\ (r),  X2 (t),  and  9(t),  and  control,  itfY),  for  configuration  #4. 
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Figure  5.9.  MCS  and  hybrid  algorithm  variance  estimates  of  states,  x\ (r),  X2 (r),  and 
0(t ),  and  control,  u(t),  for  configuration  #1. 
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Figure  5.10.  MCS  and  hybrid  algorithm  variance  estimates  of  states,  x\ (r),  X2 (r),  and 
6(t),  and  control,  u(t),  for  configuration  #2. 


Figure  5.11.  MCS  and  hybrid  algorithm  variance  estimates  of  states,  x\ (r),  x2(t),  and 
8(t),  and  control,  u(r),  for  configuration  #3. 
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Mr)],  nm 


(a)  Variance  of  Xi(t) 


(b)  Variance  of  X2  (r) 


(d)  Variance  of  u(r) 


Figure  5.12.  MCS  and  hybrid  algorithm  variance  estimates  of  states,  x\ (r),  X2 (r),  and 
6(t),  and  control,  u(t),  for  configuration  #4. 
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(d)  Covariance  of  x\  (r)  and  u(r) 


Figure  5.13.  MCS  and  hybrid  algorithm  covariance  estimates  between  the  states,  x\ (t), 
X2 (t),  and  0(t),  and  between  the  states  and  control,  w(t),  for  configuration  #1. 
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(a)  Covariance  of  x\  (r)  and  Xi  (r) 


(e)  Covariance  of  a "i{f)  and  u(t) 


(d)  Covariance  of  £i(t)  and  u{t) 


Figure  5.14.  MCS  and  hybrid  algorithm  covariance  estimates  between  the  states,  x\ (t), 
X2 (t),  and  9{t ),  and  between  the  states  and  control,  u(t),  for  configuration  #2. 
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(a)  Covariance  of  x\  (r)  and  Xi  (r) 


(e)  Covariance  of  a "i{f)  and  u(t) 


(d)  Covariance  of  £i(t)  and  u{t) 


Figure  5.15.  MCS  and  hybrid  algorithm  covariance  estimates  between  the  states,  x\ (t), 
X2 (t),  and  9{t ),  and  between  the  states  and  control,  u(t),  for  configuration  #3. 
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(a)  Covariance  of  x\  (r)  and  Xi  (r) 


(e)  Covariance  of  a "i{f)  and  u(t) 


A 


(d)  Covariance  of  a;i(r)  and  u(t) 


(f)  Covariance  of  9(t)  and  u(r) 


Figure  5.16.  MCS  and  hybrid  algorithm  covariance  estimates  between  the  states,  x\ (t), 
X2 (r),  and  8(t),  and  between  the  states  and  control,  u(t),  for  configuration  #4. 
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(a)  Best  case  threat  PDF  configuration 


xi,  NM 


(b)  Worst  case  threat  PDF  configuration 

Figure  5.17.  Trajectory  window  with  threat  rings  defined  by  POK  PDF. 
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Figure  5.18.  POK  estimates  at  each  time  point  for  best  and  worst  case  scenarios. 


Figure  5.19.  Cumulative  POK  estimates  for  best  and  worst  case  scenarios. 


129 


(a)  Best  case  threat  PDF  configuration 


xi,  NM 


(b)  Worst  case  threat  PDF  configuration 


Figure  5.20.  Trajectory  window  and  threat  rings  defined  by  POK  PDF  in  modified 
scenario. 
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Figure  5.21.  POK  estimates  at  each  time  point  for  best  and  worst  case  scenarios  using 
modified  threat  locations. 


Time,  r 


Figure  5.22.  Cumulative  POK  estimates  for  best  and  worst  case  scenarios  using  modi¬ 
fied  threat  locations. 
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Table  5.3.  Difference  between  hybrid  algorithm  and  MCS  results 


(a)  Case  #1 


(b)  Case  #2 


s 

Max  Difference 
( \SMCS  ~  Salg\ ) 

PD 

(%) 

S 

Max  Difference 

( \SmCS  ~  Salg  |) 

PD 

(%) 

Mean 

Mean 

xi(r) 

5.825E"3 

0.0262 

xi  (r) 

5.205E-3 

0.02332 

X2  (t) 

1.950E"3 

0.01027 

x2(r) 

1.217E-3 

0.003761 

9(t) 

9.214E-4 

0.09503 

0(t) 

8.404E-4 

0.07754 

u(t) 

Variance 

8.522E-2 

0.2409 

u(t) 

Variance 

0.1004 

0.4471 

Xl  (T) 

1.403E"3 

1.126 

Xi(t) 

2.002E-3 

1.138 

x2(t) 

6.815E-4 

0.901 

x2(t) 

1.228E-3 

1.143 

0(t) 

4.798E-5 

1.587 

6»(t) 

5.992E-5 

1.467 

u(t) 

Covariance 

0.4126 

1.719 

u(t) 

Covariance 

0.5126 

1.563 

xi  (r),u(r) 

2.332E-2 

2.528 

xi  (t),u(t) 

3.088E-2 

1.481 

x2  (t),u(t) 

1.702E'2 

1.275 

x2(t),u(t) 

2.496E-2 

1.373 

0(t),u(t) 

3.689E-3 

2.554 

0(t),u(t) 

4.702E-3 

3.908 

Xi(t),x2(t) 

6.25E'4 

0.7332 

xi  (t),x2(t) 

1.586E-3 

1.287 

Xi(t),0(t) 

1.638E-4 

23.92 

Xi(t),0(t) 

2.062E'4 

1.437 

x2(t),0(t) 

1.149E-4 

1.224 

x2(t),0(t) 

1.721E-4 

1.343 

MCS:  30K  samples,  387.1  minutes  pro¬ 
cessing 

MCS:  17. 5K  samples,  221.8  minutes  pro¬ 
cessing 

Hybrid  algorithm:  6.5  minutes 
ing 

(c)  Case  #3 

5  process- 

Hybrid  algorithm:  5.6  minutes  processing 

(d)  Case  #4 

S 

Max  Difference 
(\SmCS  —  Salg\ ) 

PD 

(%) 

S 

Max  Difference 

( \SmCS  ~  Salg\) 

PD 

(%) 

Mean 

Mean 

xi  (t) 

3.7E"3 

0.04095 

xi(T) 

6.796E-3 

0.03057 

x2(t) 

2.374E"3 

0.01718 

x2(r) 

1.718E-3 

0.00749 

9(t) 

8.359E'4 

0.07706 

9{t ) 

1.185E'3 

0.1093 

u(t) 

0.1379 

1.09 

u{t) 

0.1359 

1.071 

Variance 

Variance 

xi{r) 

4.977E-3 

4.002 

xi  (t) 

1.691E-3 

0.8180 

x2(t) 

3.879E-3 

4.153 

x2{t) 

9.097E-4 

1.207 

9{r) 

1.713E-4 

4.772 

9(t) 

6.581E'5 

2.021 

u(t) 

1.293 

4.55 

u{t) 

0.5254 

2.231 

Covariance 

Covariance 

xi  (t),u(t) 

7.787E-2 

4.238 

xi  (t),u(t) 

2.985E-2 

2.01 

x2(t),u(t) 

6.936E-2 

4.396 

x2(t),u(t) 

2.377E-2 

1.822 

9(t),u(t) 

7.935E"3 

4.614 

0(t),u(t) 

4.704E"3 

54.16 

x1(t),x2(t) 

4.4E-3 

4.092 

Xi(t),X2(t) 

1.214E-3 

1.228 

xi(t),0(t) 

5.251E"4 

4.193 

Xl  (t),0(t) 

2.018E-4 

1.774 

x2(t),9(t) 

4.911E'4 

4.415 

x2(t),0(t) 

1.354E-4 

1.325 

MCS:  34K  samples,  453.2  minutes  pro¬ 
cessing 

MCS:  17. 5K  samples,  234.6  minutes  pro¬ 
cessing 

Hybrid  algorithm:  6.6  minutes  process-  Hybrid  algorithm:  5.1  minutes  process- 


132 


Table  5.4.  Hybrid  algorithm  and  MCS  expected  value  and  variance  of  J 


Case 

E[J] 

Alg 

E[J] 

MCS 

PD 

(%) 

var(J) 

Alg 

var(J) 

MCS 

PD 

(%) 

1 

1.6156E"5 

1.6160E"5 

2.4756E'2 

3.5976E'13 

3.5961E"13 

4.1703E"2 

2 

1.6136E"5 

1.6155E"5 

0.1177 

4.8037E"13 

4.9090E"13 

2.1683 

3 

1.6148E"5 

1.6144E"5 

2.4774E"2 

3.6575E"13 

3.7227E"13 

1.7669 

4 

1.6162E"5 

1.6163E"5 

6.1872E"3 

4.7421E"13 

4.7438E"13 

3.5843E"2 

Deterministic  solution  of  J:  1.6164E"5 


Table  5.5.  Hybrid  algorithm  and  MCS  expected  value  and  vari¬ 
ance  of  tf 


Case 

Alg 

EM 

MCS 

PD 

(%) 

var(tf) 

Alg 

yar  (tf) 
MCS 

PD 

(%) 

1 

0.1231 

0.1231 

0 

1.5542E"7 

1.5493E"7 

0.3158 

2 

0.1231 

0.1231 

0 

2.0915E"7 

2.0793E"7 

0.5850 

3 

0.1231 

0.1231 

0 

1.6921E"7 

1.7553E"7 

3.6665 

4 

0.1231 

0.1231 

0 

1.8987E"7 

1.9238E"7 

1.3133 

Deterministic  solution  of  tf.  0.1230  hours 
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VI.  Conclusions  and  Recommendations 


A  hybrid  numerical  algorithm  was  presented  in  this  dissertation  which  combines 
the  GPM  with  the  collocation  form  of  the  gPC  method  to  quantify  the  effects 
of  stochastic  parameters  on  OC  and  TO  problem  solutions.  The  GPM  and  gPC 
have  both  been  shown  to  be  spectrally  accurate  numerical  methods  for  solving  deter¬ 
ministic  optimal  control  problems  and  stochastic  differential  equations,  respectively. 
The  gPC  method  samples  the  random  space  using  collocation  nodes,  which  are  then 
inserted  into  the  differential  equations  creating  a  set  of  deterministic  differential  equa¬ 
tion  problems  that  are  solved  using  standard  ODE  and  PDE  solvers.  The  resulting 
set  of  deterministic  solutions  are  used  in  stochastic  computations  to  characterize  the 
distribution  of  the  solution  by  constructing  a  polynomial  representation  of  the  out¬ 
put  as  a  function  of  uncertain  parameters.  If  OC  and  TO  problems  are  considered  in 
the  context  of  the  HBVP  formulation,  an  indirect  transcription  of  the  optimization 
problem  into  a  differential  equation  problem,  and  the  GPM  is  thought  of  as  a  nu¬ 
merical  method  that  produces  an  approximate  solution  to  the  optimization  problem 
that  is  equivalent  to  an  approximate  pseudospectral  solution  to  the  HBVP,  then  it  is 
reasonable  to  theorize  that  a  numerical  GPM  solver  such  as  GPOPS  could  be  used  in 
place  of  the  deterministic  differential  equation  solvers  traditionally  used  in  the  gPC 
algorithm.  Thus,  the  hybrid  algorithm  considered  in  this  document  integrates  the 
GPOPS  deterministic  OC  problem  solver  into  the  gPC  construct  thereby  providing 
a  set  of  spectrally  accurate  numerical  solutions  to  the  sampled  problems  that  satisfy 
boundary,  path,  and  bounded  state  and  control  constraints  while  minimizing  the  cost 
functional  and  are  suitable  for  stochastic  computations,  thus  extending  the  current 
and  well-established  gPC  method  of  solving  SDEs  to  be  applicable  to  TO  and  OC 
problems.  The  goal  of  the  research  presented  in  this  document  was  to  demonstrate 
that  the  hybrid  algorithm  combining  the  GPM  and  gPC  methods  can  effectively  be 
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applied  to  quantify  the  effects  of  uncertainty  in  stochastic  optimization  problems. 


6.1  Conclusions 

The  hybrid  GPM-gPC  algorithm  was  applied  to  two  nonlinear  concept  demonstra¬ 
tion  problems  to  demonstrate  the  ability  of  the  hybrid  algorithm  to  solve  nonlinear 
optimization  problems  with  uncertain  parameters. 

The  first  problem,  an  adaptation  of  a  textbook  problem,  was  an  OC  problem 
with  nonlinear  dynamics  and  cost  functional,  fixed  final  state  and  time,  and  multi¬ 
plicative  Gaussian  uncertainties  effecting  the  state  variable  dynamics  equations.  The 
solution  to  the  problem  was  a  set  of  polynomial  functions  approximating  state  vari¬ 
ables,  control,  and  cost  solutions  whose  expected  values,  variances,  and  covariances, 
were  found  using  the  expansion  coefficients  to  describe  the  statistical  properties  of 
the  solution.  Comparing  the  statistical  properties  determined  by  application  of  the 
hybrid  algorithm  with  the  MCS  results  demonstrated  close  agreement  between  the 
two  methods.  The  error  differences  between  the  two  methods  did  however  bring  into 
question  whether  closer  agreement  would  result  if  either  the  gPC  parameters,  number 
of  collocation  points  and  order  of  polynomial  approximation,  or  the  MCS  convergence 
tolerance  were  adjusted.  A  second  MCS  with  lower  convergence  tolerance  was  con¬ 
ducted  resulting  in  differences  between  the  two  methods  decreasing  by  28  to  67%, 
leading  to  the  conclusion  that  the  hybrid  algorithm  was  actually  providing  more  ac¬ 
curate  approximations.  The  results  presented  show  that  the  hybrid  algorithm  is  able 
to  quantify  the  effects  the  Gaussian  parameters  on  the  optimal  solution  of  a  very 
challenging  nonlinear  problem  while  requiring  dramatically  fewer  sample  points  and 
associated  GPOPS  deterministic  solutions  and  computation  time. 

The  second  concept  demonstration  problem  was  designed  to  be  representative  of 
a  real-world  mission  scenario  that  has  USSTRATCOM  interest.  The  scenario  was 
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constructed  to  find  the  path  through  a  two-dimensional  space  that  minimizes  the 
POK  by  lethal  threats  whose  locations  are  uncertain,  to  quantify  the  effects  those 
uncertainties  have  on  the  trajectory  solution  by  estimating  the  statistical  properties, 
and  to  use  the  statistical  properties  to  estimate  the  probability  of  the  vehicle  being 
killed  during  the  mission,  ft  is  a  TO  problem  that  is  nonlinear  in  the  state  equa¬ 
tions  and  cost  functional,  has  fixed  final  state  and  free  final  time  specifications,  and 
incorporates  additive  uncertainties  that  effect  the  cost  functional.  Four  cases  were 
evaluated  where  the  uncertainties  in  the  threat  locations  were  modeled  as  random 
variables  with  either  all  Gaussian,  all  uniform,  or  a  mix  of  the  two  distributions. 
Similar  to  the  first  concept  demonstration  problem,  the  algorithm  generated  a  set  of 
polynomial  approximation  functions  for  the  state  variables,  control,  cost,  and  termi¬ 
nal  time  whose  expected  values,  variances,  and  covariances,  were  calculated  using  the 
expansion  coefficients  to  describe  the  statistical  properties  of  the  solution.  Compar¬ 
ing  the  statistical  properties  determined  by  application  of  the  hybrid  algorithm  with 
the  MCS  results  again  demonstrated  that  the  hybrid  algorithm  effectively  quantifies 
the  effects  of  uncertainty  with  comparable  accuracy  to  the  MCS  while  requiring  dra¬ 
matically  fewer  sample  points,  GPOPS  calls,  and  computation  time.  Additionally, 
mixing  probability  distributions,  and  thus  polynomial  bases,  did  not  seem  to  cause 
any  appreciable  change  in  algorithm  performance  requiring  modifications  to  numbers 
of  collocation  points  or  polynomial  order,  or  unfavorable  comparison  with  the  MCS 
results.  Probability  of  kill  computations,  which  really  was  a  repackaging  of  the  hybrid 
algorithm’s  statistical  data  outputs,  demonstrated  that  the  results  can  be  presented 
in  a  form  that  can  be  used  by  mission  planners  and  aircrews  to  asses  risks  associated 
with  a  mission  trajectory  solution. 

Considering  that  two  types  of  nonlinear  optimal  control  problems  were  investi¬ 
gated  with  fixed  boundary  conditions  in  the  first  problem  and  a  mix  of  fixed  and  free 
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boundary  conditions  in  the  second,  and  with  uncertainties  effecting  different  equa¬ 
tions  in  the  problem  formulations,  it  is  apparent  that  inserting  the  GPM  into  the  gPC 
construct  results  in  a  new  algorithm  that  is  capable  of  solving  a  variety  of  nonlin¬ 
ear  optimal  control  problems  with  uncertain  parameters.  The  results  presented  have 
demonstrated  that  the  method  is  able  to  generate  solutions  to  the  stochastic  TO 
and  OC  problems  by  determining  the  expected  value  solutions  for  the  state  variables, 
control,  cost,  terminal  time  and  characterize  the  probabilistic  information  about  the 
solutions  that  agrees  with  MCS  results,  thus  characterizing  how  the  optimal  solu¬ 
tion  changes  with  uncertainty.  Therefore,  the  hybrid  GPM-gPC  algorithm  developed 
herein  extends  gPC  methods  by  providing  a  tool  with  great  potential  for  uncertainty 
quantification  in  optimal  control  and  trajectory  optimization  problems. 

6.2  Recommendations  for  Future  Research 

The  concept  demonstration  problems  considered  in  this  work  were  designed  to 
be  manageable  within  the  confines  of  the  limited  processing  capabilities  that  were 
available.  The  results  presented  indicate  that  processing  requirements  in  terms  of 
computation  time  and  number  of  sample  points  required  to  solve  the  demonstration 
problems  was  reasonable,  but  slight  changes  to  these  problems,  such  as  adding  addi¬ 
tional  state  equations,  increasing  the  number  of  random  dimensions,  or  widening  the 
PDF  windows  of  the  uncertain  parameters,  causes  dramatic  increases  in  computa¬ 
tional  burden  that  quickly  exceeds  the  capabilities  of  available  resources.  Therefore, 
further  investigation  should  first  focus  on  improving  computational  efficiency  of  the 
algorithm  by  determining  guidelines  for  choosing  the  numbers  of  polynomial  basis 
elements  combined  with  incorporation  of  sparse  grids  to  reduce  the  number  of  collo¬ 
cation  points  and  GPOPS  calls.  Calling  GPOPS  441  times  in  the  first  problem,  and 
343  times  in  the  second,  while  using  fifth-  and  seventh-order  polynomial  bases,  respec- 
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tively,  was  not  overly  time  consuming,  but  keeping  in  mind  that  GPOPS  uses  an  NLP 
solver,  which  is  an  iterative  routine  in  itself,  to  calculate  state  and  control  solutions 
that  meet  optimality  conditions,  it’s  obvious  that  as  the  number  of  states,  controls, 
and  random  inputs  grow  the  computational  burden  will  increase  exponentially. 

Once  efficiency  improvements  have  been  considered,  further  research  should  be 
conducted  using  a  high-performance  workstation  with  multiple  processors  and  higher 
memory  capacity  enabling  expansion  of  the  problem  formulations.  Although  GPOPS 
is  not  written  for  parallel  processing,  the  hybrid  algorithm  code  evaluating  the  ex¬ 
pansion  coefficients  and  output  functions  was  able  to  take  advantage  of  the  avail¬ 
able  multi-core  processor  and  parallel  processing  functionality  in  MATLAB®to  pro¬ 
vide  noticeable  improvement  in  processing  time.  Further  implementation  on  a  multi¬ 
processor  computer  is  expected  to  provide  even  more  improvements  in  processing 
time  and  enable  problem  expansions.  The  first  problem  would  be  suitable  for  con¬ 
sidering  time-varying  random  parameters  effecting  the  state  equations.  Further  de¬ 
veloping  the  algorithm  to  quantify  the  effects  of  time-varying  uncertainties  such  as 
wind  gusts  or  sensor  noise  could  eventually  lead  to  applying  the  hybrid  algorithm  to 
real-  or  near-real-time  state  estimation  and  control  problems.  The  second  problem 
should  be  expanded  by  adding  three  additional  random  elements  to  more  realistically 
model  potential  threat  movement  in  the  two-dimensional  space.  This  expansion  was 
attempted,  but  using  the  same  seven  points,  seventh-order  polynomial  bases,  and  ten¬ 
sor  product  spaces  resulted  in  the  number  of  collocation  points  increasing  from  343  to 
117,649  and  the  number  of  basis  polynomials  increasing  from  512  to  262,144,  which 
was  beyond  the  capabilities  of  available  computers.  On  a  more  powerful  computer, 
additional  threats  could  also  be  added  as  desired  to  represent  other  mission  scenarios 
USSTRATCOM  may  encounter. 

With  improved  computational  efficiency,  high-performance  computer  resources, 
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and  algorithm  enhancements  to  solve  problems  with  time- varying  random  inputs,  the 
algorithm  could  then  be  applied  to  MAV  research.  MAV  control  problems  are  chal¬ 
lenging  because  the  small,  lightweight  aircraft  are  inherently  nonlinear  and  vulnerable 
to  exogenous  disturbances  such  as  wind  gusts  and  turbulence.  Vehicle  models  are  still 
being  developed  through  theoretical  analysis  and  both  wind  tunnel  and  flight  test¬ 
ing  since  traditional  aerodynamic  theory  does  not  adequately  describe  low  Reynolds 
number  aerodynamics.  Low-cost  sensors  used  on  these  vehicles  have  more  measure¬ 
ment  noise  and  errors  than  higher-fidelity  sensors.  Therefore,  MAV  problems  are 
stochastic  in  nature  with  model  uncertainties,  measurement  noise,  and  time-varying 
random  disturbances.  The  hybrid  algorithm  may  provide  an  effective  tool  to  solve 
nonlinear  MAV  optimization  problems  while  including  these  stochastic  elements. 

Applying  the  hybrid  GPM-gPC  algorithm  to  other  classes  of  nonlinear  OC  prob¬ 
lems  with  uncertain  parameters  is  also  recommended  as  future  research.  Two  types 
of  problems  where  the  hybrid  algorithm  may  be  effective  in  quantifying  the  effects 
of  uncertain  parameters  on  optimal  solutions  are  bifurcation  and  chaotic  problems. 
Bifurcation  occurs  in  a  dynamical  system  when  a  “quantitative  change  of  parameters” 
leads  to  a  “qualitative  change  in  system  properties”  [76].  In  the  context  of  applying 
the  hybrid  algorithm  to  stochastic  OC  problems,  uncertain  parameters  effecting  the 
state  equations  may  result  in  a  qualitative  change  of  system  properties  where  solutions 
to  the  set  of  deterministically  sampled  problems  used  in  stochastic  computations  may 
result  in  clustering  around  more  than  one  apparent  expected  value.  This  clustering 
would  indicate  discontinuity  in  the  functional  relationship  between  the  approximate 
solution  and  the  uncertain  parameters,  suggesting  that  effective  application  of  the 
hybrid  algorithm  may  require  a  method  for  determining  the  points  in  the  finite  prob¬ 
ability  space  where  bifurcation  occurs  and  application  of  the  hybrid  algorithm  over 
each  corresponding  sub-domain. 
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The  second  class  of  nonlinear  OC  problems  involves  applying  the  hybrid  algorithm 
to  chaotic  systems  where  uncertainty  is  included.  Chaotic  behavior  is  seen  in  strongly 
nonlinear  systems  and  is  characterized  by  “output  [s]  that  are  extremely  sensitive  to 
initial  conditions”  where  very  small  changes  in  the  initial  conditions  result  in  radically 
different  state  trajectories  [76].  Optimal  control  problems  could  be  considered  where 
uncertainties  are  incorporated  in  initial  conditions  or  in  both  initial  conditions  and 
parameters  effecting  the  dynamics  equations  to  investigate  the  ability  of  the  hybrid 
algorithm  to  quantify  the  effects  of  the  uncertainties  on  the  solutions.  A  comparison 
should  be  made  between  the  output  approximation  function  evaluated  at  several  test 
points  drawn  from  the  assumed  probability  distributions  of  the  uncertain  parameters 
and  deterministic  solutions  based  on  those  test  points  to  verify  the  accuracy  of  the 
hybrid  algorithm  approximation.  Additionally,  since  system  responses  are  extremely 
sensitive  to  small  changes  in  initial  conditions,  a  convergence  analysis  should  be  per¬ 
formed  to  determine  if  exponential  convergence  is  preserved  since  the  approximate 
solutions  may  not  be  smoothly  dependent  on  the  uncertain  parameters.  Investigating 
applying  the  hybrid  GPM-gPC  algorithm  to  these  two  classes  of  nonlinear  problems 
in  conjunction  with  the  previously  recommended  research  will  further  extend  the 
gPC  method  to  be  a  useful  tool  in  determining  the  effects  uncertainties  have  on  OC 
problem  solutions. 

The  work  presented  in  this  document  is  a  first  step  in  combining  GPM  and  gPC 
numerical  methods  to  consider  nonlinear  optimal  control  problems  effected  by  stochas¬ 
tic  elements.  The  results  have  demonstrated  that  the  hybrid  algorithm  is  capable  of 
quantifying  the  effects  of  uncertainties  in  problems  that  are  meaningful  to  the  USAF 
and  shows  great  potential  for  further  applications  to  solve  challenging  control  prob¬ 
lems. 
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Appendix  A.  Hybrid  GPM-gPC  Code 


Selected  lines  of  MATLAB®code  are  given  in  this  appendix  to  show  how  the  steps 
of  the  hybrid  GPM-gPC  algorithm  presented  in  Chapter  111  were  implemented 
to  solve  the  TO  problem  in  Chapter  V.  Similar  code  was  used  to  produce  the  results 
presented  for  the  OC  problem  in  Chapter  IV. 


1.1  Collocation  Nodes 

Step  #1:  Calculate  collocation  nodes  and  quadrature  weights  {pj,aj}j=l: 

Listing  A.l.  Calculate  collocation  nodes  and  quadrature  weights 

1  Define  constants  needed  to  set  up  the  TO  problem 
constants  .  v  =  470  ;  °/0  Cruise  speed  in  Kt 

constants  .  ft2Nm  =  l/6076 . 1 1549  ;  °/0  Convert  ft  to  nautical  miles 

const  ant  s  .  deg2rad  =  pi  /  180  ;  °/«  Convert  degrees  to  radians 

const  ant  s  .  Rmin  =  1 1 326  *  const  ant  s  .  f  t2Nm  ;  °/«  Assumed  turn  radius  ... 

converted  to  NM 

*/.  Properties  of  the  2-variate  Gaussian  PDF  used  to  model  threat  . 
rings 

constants . rhol =0 . 0 ;  constants . rho2=0 . 0 ;  constants . rho3=0 . 0 ; 
const  ant s . s igt 1 x 1 =5 ;  constants . sigt2xl=5 ;  constants . sigt3xl=5 ; 
constants . sigtlx2=5;  constants . sigt2x2=5 ;  constants . sigt3x2=5 ; 

•/.  Center  locations  of  threats 
constants . clxl=10;  constants . clx2=30; 
constants . c2xl=25;  constants . c2x2=15; 
constants . c3xl=35;  constants . c3x2=30; 

N  =  3;  '/.  Specify  number  of  random  dimensions  (random  parameters) 
q  =  7 ;  7„  Specify  number  of  collocation  points  in  each  random  ... 
dimension 

Q=q~N;  ”/0  Total  number  of  collocation  points 

dl=7;  d2  =  7 ;  d3=7;  P  =  21;  °/0  1-D  polynomial  order  (d_i)  and  N-D  ... 
polynomial  overall  order  (P) 

mu  =  0;  dev  =  0.5;  °/n  Define  properties  of  Gaussian  distributions 
*/.  For  Gaussian  distributions 

[a  ,  wa]  =hermquadpt  s  (  ql )  ;  "/,  hermquadpts  function  from  Mathworks 

website 

[b , wb] =hermquadpts (q2)  ; 

[c,wc]=hermquadpts(q3)  ; 


°/0  scale  points 


a=a . *  sqrt (2*dev~2) +mu ; 
b=b . *  sqrt (2*dev~2) +mu ; 
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c=c . *sqrt (2*dev~2) +mu ; 

wa=wa  .  /  sqrt  ( pi  )  ;  7«  scale  quadrature  weights 

wb=wb . / sqrt ( pi ) ; 
wc=wc ./ sqrt (pi) ; 

7,  For  unif orm  distributions 

[a  ,  wa]  =  1  egquadpt  s  (  ql )  ;  7«  legquadpts  function  from  Mathworks 

website 

[b , wb] = 1 egquadpt s ( q2 )  ; 

[c,wc]=legquadpts(q3)  ; 

lb=-l;  %  lower  bound  of  uniform  distribution 
ub  =  l;  %  upper  bound  of  uniform  distribution 


a=((a./2)  +(1/2)) .*(ub-lb)+lb 
b=((b./2)  +(1/2) )  ,*(ub-lb)+lb 
c=((c./2)  +(1/2))  ,*(ub-lb)+lb 


7,  scale  points 


wa=wa . / sum (wa) 
wb=wb . / sum ( wb ) 
wc=wc . / sum (wc ) 


7.  scale  weights 


"/,  Construct  tensor  product  point  and  weight  sets 
count  =  1 ; 
for  ii=l : ql 

for  j  j  = 1 : q2 

for  kk=l : q3 

qc  (  count  ,  :  )  =  [a  (  i  i  )  ,  b  (  j  j  )  ,  c  (  kk )  ]  ;  7«  array  of  Q  points 

wt  c  (  count  ,  :)  =  [wa(ii)  ,wb(jj)  ,wc(kk)]  ; 

wcprod ( count ) =wa ( i i ) *wb ( j j ) *wc ( kk) ;  %  array  of  Q  weights 

count  =  count  + 1  ; 

end 


end 


end 


1.2  Apply  GPOPS  to  Deterministic  Sample  Problems 


Step  #2:  Using  each  of  the  Q  collocation  points  Pj,  solve  the  set  of  deterministic 
OC  problems  using  GPOPS: 


Listing  A. 2.  Solve  Q  deterministic  problems  using  GPOPS 


for 


ii=l : Q 

7.  Seperate  samples  for  each  dynamics  equation 
ac  =  qc ( ii  ,  1 )  ; 
bc  =  qc ( ii ,2)  ; 
cc  =  qc ( ii  ,3)  ; 
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7.  Call  the  main  GPOPS  script 
MyPr oblemMain ; 

°/o  Save  state  trajectory  data 
xldata= [xldata ; xlout ’] ; 
x2data= [x2data ; x2out ’] ; 
x3data= [x3data ; x3out ’] ; 

7,  Save  control  data 
udata= [udata ; uout  ’]  ; 

7.  Save  cost  data 
cost= [cost ; costout] ; 

7.  Save  time  data 
tdata= [tdata ; tout  ’]  ; 


end 


1.3  Expansion  Coefficients 

Step  #3:  Evaluate  the  expansion  coefficients  using  (3.1): 


Q 


(m  =  1, M) 


Listing  A. 3.  Evaluate  expansion  coefficients 
7.  Build  the  Hermite  polynomial  (H)  or  Legendre  polynomial  (P)  ... 

basis  functions  and  evaluate  at  all  combinations  of  a,  b,  and  c 

7.  Calculate  polynomial  normaliztion  factors  from  Xiu  1  s  book  [86] 

7.  For  Gaussian  distributions 
for  ii=0 : dl 


intl(ii  +  l)=factorial(ii)  ; 


end 


for  i i =0 : d2 


int2(ii+l)=factorial(ii) ; 


end 


for  i i =0 : d3 


int3(ii+l)=factorial(ii) ; 


end 


7.  For  uni f orm  distributions 
for  ii=0 : dl 


inti (ii+1) =1/ (2*ii+l) ; 


end 


for  i i =0 : d2 
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int2 (ii+1) =l/(2*ii+l) ; 

end 

for  i i =0 : d3 

int3(ii+l)=l/ (2*ii+l) ; 

end 

7,  Build  polynomial  bases 

7«  For  Gaussian  distributions  ...  use  Hermite  polynomial  basis  ... 

functions  (hermite  function  from  Mathworks  website) 
for  ii=l:Q 
m=  1  ; 

for  j  j  =0 : dl 

for  kk=0 : d2 

for  11 =0 : d3 

hi ( ii , m)  =  ( 1 / ( ( 2) ~ ( j  j /2) ) ) thermit e ( j  j  ,(qc(ii,l) -mu) / ( . . 

sqrt (2*dev~2) ) ) / sqrt ( inti (jj  +1) )  ; 
h.2  ( ii  ,  m)  =  ( 1  /  ( ( 2)  ~  ( kk/2)  ) )  *hermit e  (kk  ,  ( qc  ( ii  ,  2) -mu) / ( . . 

sqrt (2*dev~2) ) ) / sqrt (int2 (kk+1) ) ; 
h.3  (  ii  ,  m)  =  (  1  /  (  (  2)  ~  (  11  /  2)  )  )  *hermit  e  ( 11  ,  (  qc  (  ii  ,  3) -mu) / ( . . 
sqrt (2*dev~2) ) ) / sqrt (int3 (11+1) ) ; 

if  j  j  +kk  +  ll <  =  P 

Phi(ii,m)=hl(ii ,m)*h2(ii ,m)*h3(ii,m) ; 
m=m+ 1 ; 

end 

end 

end 

end 

end 

7,  For  uniform  distributions  ...  use  Legendre  polynomial  basis  ... 

functions  ( Legendr ePoly  function  from  Mathworks  website) 
for  ii=l:Q 
m  =  1  ; 

for  j  j  =0 : dl 

for  kk=0 : d2 

for  11 =0 : d3 

hl(ii  ,m)=polyval(LegendrePoly(jj)  ,2*((qc(ii ,  1 ) - lb )  / (ub 
-lb)  ) -1)/ sqrt (inti (jj+1))  ; 

h2 ( ii ,m)=polyval(LegendrePoly (kk) ,2*((qc(ii , 2) - lb )  / (ub 
-lb) ) -1) /sqrt ( int2 (kk+1) ) ; 

h.3  (  ii  ,m)=polyval(LegendrePoly(ll)  ,2*((qc(ii  ,  3)  -  lb  )  /  (ub 
-lb)) -l)/sqrt(int 3(11+1)) ; 

if  j  j  +kk  +  ll <  =  P 

Phi ( ii ,m)=hl (ii , m ) *h2(ii , m) *h3(ii ,m) ; 
m=m+ 1 ; 

end 

end 

end 

end 
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end 


[rphi  cphi  ]  =  s  ize  (  Phi  )  ; 

M=  cphi ; 

7,  Calculate  expansion  coefficients 
for  ii=l : M 

parfor  j  j  =  1  :  Q  7.  Parallel  processing  speeds  up  evaluation  of 
expansion  coefficients 
tt=Phi(jj ,ii)*wcprod(jj) ; 


zj ( j j ) =cost ( j j ) *tt ; 
z't(jj)='tdata(jj  ,Nnodes+2)*tt; 

zl(jj  ,  : ) =xldata ( j j  ,  :)  .  *tt  ; 
z2(jj , :)=x2data(jj , :) . *tt ; 
z3(jj , :)=x3data(jj , :) . *tt ; 
zu(jj , : ) =udata ( j j , :) .  *tt  ; 
end 

7.  Expansion  coefficients  for  states  x_l  ,  x_2  ,  and  x_3 
zhatl(ii,:)= sum ( wl ) ; 
zhat2(ii,:)= sum (w2 ) ; 
zhat3(ii,:)  =  sum (w3 )  ; 

7.  Expansion  coefficients  for  control  u 
zhatu(ii ,  : )  =  sum (wu)  ; 

l  Expansion  coefficients  for  cost  J 
zhat  j  (ii ,  :)  =  sum (w j )  ; 

%  Expansion  coefficients  for  terminal  time  t_f 
zhatt (ii , : )= sum ( wt ) ; 


end 


1.4  Output  Approximation  Function 

Step  #4:  Build  the  output  approximation  function  using  (3.2): 


M 


m=  1 


Listing  A. 4.  Build  output  approximation  function 

7.  Construct  polynomials  and  evaluate  points  of  interest  a_out  , 
b_out ,  and  c_out 

7.  For  Gaussian  distribut  ions  ...  use  Hermite  polynomial  basis  .. 
functions  (hermite  function  from  Mathworks  website) 
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m  =  l  ; 

for  j  j  =0 : dl 

for  kk=0 : d2 

for  11=0: d3 

hll(m)  =  (l/((2)~(jj/2)))*h.ermite(jj  ,  (aout  -  mu )  /  sqrt  (2*dev. 
~2))/sqrt(intl(jj+l)) ; 

h22(m)=(l/ ((2)~(kk/2)))*hermite(kk, (bout -mu ) / sqrt ( 2*  dev . 
~2) ) /sqrt ( int2 (kk+1) ) ; 

h33(m) =(1/ ( (2) ~ (11/2) ) ) *hermite(ll , (cout -mu ) / sqrt (2*dev. 
~2)) /sqrt (int3 (11+1)) ; 

if  j  j  +kk  +  ll <  =  P 

Phi out (m) =hl 1 (m) *h22 (m) *h33 (m) ; 
m=m+ 1 ; 

end 

end 

end 

end 

7,  For  uniform  distributions  ...  use  Legendre  polynomial  basis  ... 

functions  ( Legendr ePoly  function  from  Mathworks  website) 
m  =  1  ; 

for  j  j  =0 : dl 

for  kk=0 : d2 

for  11=0: d3 

hll(m)=polyval(LegendrePoly(jj) , aout) /sqrt (inti (jj+1)) ; 
h22 (m) =polyval ( LegendrePoly (kk) , bout )/ sqrt ( int2 (kk+ 1 ) ) ; 
h33 (m) =polyval ( Legendr ePo ly ( 11 ) , cout )/ sqrt ( int3 ( 11 + 1 ) ) ; 

if  j  j  +kk  +  ll <  =  P 

Phiout (m)=hll (m) *h22(m) *h33 (m) ; 
m=m+ 1 ; 

end 

end 

end 

end 

parfor  ii  =  l:M  7,  Parallel  processing  speeds  up  evaluation  of  output 
approximation 

zl(ii,  :)=zhatl(ii  ,  :)  . *Phiout(ii)  ; 
z2(ii,  :)=zhat2(ii  ,  :)  .*Phiout(ii)  ; 
z3(ii,  :)=zhat3(ii  ,  :)  .*Phiout(ii)  ; 
zu(ii,  :)=zhatu(ii,  :)  .*Phiout(ii)  ; 
zj (ii)=zhatj (ii)*Phiout (ii) ; 
zt (ii)=zhatt (ii) *  Phi  out (ii) ; 

end 

7.  Output  approximation  of  states  x_l,  x_2  ,  and  x_3 
zout  1  =  sum  (  z  1 )  ; 
zout  2  =  sum  (  z2  )  ; 
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zout3=sum(z3) ; 

7.  Output  approximation 
zoutu= sum ( zu) ; 

of 

control  u 

7.  Output  approximation 
zout  j  =  sum  (  z  j  )  ; 

of 

cost  J 

7.  Output  approximation 
zoutt  =  sum ( zt )  ; 

of 

terminal  time  t_f 

1.5  Statistics  of  the  Solution 


Step  7^5:  Evaluate  the  statistics  of  the  approximate  solution: 


•  Expected  value  (3.3): 


E(z(t))  «E(z£(t)) 


M 


y  zm(t)$m(p) 


m= 1 


p(p)dp 


Zi(t) 


•  Variance  (3.4): 


M 

var  [z (t)}  «  ^ 

m= 2 


•  Covariance  (3.5): 


M 

COV  [ Zi(t),Zj(t )]  Ri  y  [Zi,m(t)zj!m(t)] 
m=2 


Listing  A. 5.  Evaluate  statistics 
7„  Expected  values  of  hybrid  algorithm  solution 
7,  Expected  value  of  states  x_l,  x_2  ,  and  x_3 
MeanXlgPC  =  zhatl (1  ,  :)  ; 

MeanX2gPC  =  zhat2 (1  ,  : )  ; 

MeanX3gPC  =  zhat3 (1  ,  : )  ; 

7.  Expected  value  of  control  u 
MeanUgPC  =  zhatu ( 1  ,  : )  ; 

7.  Expected  value  of  cost  J 
Mean JgPC  =  zhat j  (1)  ; 

7.  Expected  value  of  terminal  time  t_f 
Meantf gPC=zhatt (1) ; 

7.  Expected  values  of  MCS  solution 
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7,  Expected  value  of  states  x_l,  x_2  ,  and  x_3 
MeanXlMC=mean (xlMC) ; 

MeanX2MC=mean (x2MC) ; 

MeanX3MC=mean (x3MC) ; 

°i  Expected  value  of  control  u 
MeanUMC=mean (uMC) ; 

7«  Expected  value  of  cost  J 
Mean JMC=mean ( costMC ) ; 

7«  Expected  value  of  terminal  time  t_f 
Meantf  =mean(tMC(:  , end))  ; 

7,  Variances  of  hybrid  algorithm  solution 
7.  Variances  of  state  estimates  x_l  ,  x_2  ,  and  x_3 
VarXlgPC  =  sum (zhatl(2:end,  :)  .  ~  2 )  ; 

VarX2gPC  =  sum (zhat2(2:end,  :)  . " 2 )  ; 

VarX3gPC  =  sum (zhat3(2:end,  :)  .  ~ 2 )  ; 

7o  Variances  of  control  estimate  u 
VarUgPC=sum (zhatu(2:end,  :)  .  “  2)  ; 

7.  Variances  of  cost  estimate  J 
Var JgPC=sum (zhatj (2: end) . “ 2) ; 

7.  Variances  of  terminal  time  estimate  t_f 
VartfgPC  =  sum (zhatt (2: end)  .  ~  2)  ; 

7,  Variances  of  MCS  solution 

7.  Variances  of  state  estimates  x_l,  x_2  ,  and  x_3 
VarXlMC=var (xlMC) ; 

VarX2MC=var (x2MC) ; 

VarX3MC=var (x3MC) ; 

7.  Variances  of  control  estimate  u 
VarUMC=var (uMC) ; 

7.  Variances  of  cost  estimate  J 
Var JMC=var ( costMC ) ; 

7.  Variances  of  terminal  time  estimate  t_f 
VartfMC=var(tMC(: , end) ) ; 

7.  Covariance  of  hybrid  algorithm  solution 
7»  Covariances  between  state  estimates  x_l,  x_2  ,  and  x_3 
CovXlX2gPC  =  sum (whatl  (2:end,  :)  ,*what2(2:end,  :))  ; 

CovXlX3gPC  =  sum ( what  1 (2 : end ,  : )  . *  what 3 ( 2 : end ,  : ) )  ; 

CovX2X3gPC  =  sum ( what 2 (2:end,  :)  .*what3(2:end,  :))  ; 

7.  Covariances  between  state  estimates  x_l  ,  x_2  ,  and  x_3  and 
control  estimate  u 

CovXlUgPC  =  sum ( what  1 ( 2 : end ,  :)  .*whatu(2:end,  :))  ; 

CovX2UgPC  =  sum ( what 2 (2:end,  :)  .*whatu(2:end,  :))  ; 

CovX3UgPC  =  sum ( what 3 (2:end,  :)  .*whatu(2:end,  :))  ; 

7.  Covariances  of  MCS  solution 
for  i i = 1 : QMC 

for  j j = 1 : numt ime 

cl  (  ii  ,  j  j ) =xlMC ( ii , j  j ) -MeanX 1MC ( j  j )  ; 
c2 (ii , jj ) =x2MC (ii , jj ) -MeanX2MC (jj ) ; 
c3 (ii , jj ) =x3MC (ii , jj ) -MeanX3MC (jj ) ; 
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c4(ii,jj) =uMC ( ii , j  j ) -MeanUMC ( j  j )  ; 

end 

end 

1  Covariances  between  state  estimates  x_l ,  x_2 ,  and  x_3 

CovXlX2MC=mean ( cl ,*c2) ; 

CovXlX3MC=mean(cl .  *  c3) ; 

CovX2X3MC=mean(c2.*c3) ; 

l  Covariances  between  state  estimates  x_l,  x_2 ,  and  x_3  and  ... 

control  estimate  u 
CovXlUMC=mean(cl . *  c4) ; 

CovX2UMC=mean(c2.*c4) ; 

CovX3UMC=mean(c3.*c4) ; 

The  code  in  this  appendix  forms  the  basis  for  a  main  MATLAB®script  that  was 
used  that  calls  GPOPS  (Appendix  B)  and  MCS  (Appendix  C)  routines  where  indi¬ 
cated. 
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Appendix  B.  GPOPS  Set-Up  Code 


he  GPOPS  set-up  files  used  to  produce  results  presented  in  Chapter  V  are 


shown  in  this  appendix.  This  representative  code  shows  the  GPOPS  configu¬ 


ration  used  to  produce  solutions  at  the  collocation  points  (hybrid  algorithm)  and  at 
the  randomly  sampled  points  (MCS).  See  [72]  for  detailed  descriptions  about  these 


files. 


2.1  GPOPS  Main  Script:  MyProblemMain.m 


Listing  B.l.  GPOPS  main  script 


clc 

clear  setup  limits  guess 

global  qc  ac  be  cc  Nnodes  constants  t  PK  xinit  xfinal 

xlO  =  0; 
xlf  =  25; 

x20  =  xinit (2)  ; 
x2f  =  xf inal  (2)  ; 

x30  =  at  an ( x2f /xlf )  ; 
x3f  =  []  ; 

xlmin  =  0; 
xlmax  =  50; 

x2min  =  xlmin; 
x2max  =  xlmax ; 

[xgrid , ygrid ,PK]=killrings(constants  , ac ,bc , cc , xlmin , xlmax ,x2min , . .  . 
x2max ) ; 

constants . tmin=0 ; 
constants . tmax=l ; 

x3min  =  -2*pi ; 
x3max  =  2*pi  ; 

tinit  =0 ; 
tf  inal  =  []  ; 
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param_min  =  []  ; 

param.max  =  []  ; 

path_min  =  []  ; 

path_max  =  []  ; 

event.min  =  [xlO;  x20 ;  x30;  xlf;  x2f]; 
event.max  =  [xlO;  x20 ;  x30;  xlf;  x2f]; 

dur at i on_min  =  constants . tmin ; 
duration_max  =  constants . tmax ; 

iphase  =  1 ; 

limits ( iphase ). nodes  =  Nnodes  ;  ”/«  60; 


limits ( iphase ) 

t ime . min 

= 

[0 

constant  s 

tmin] 

limits ( iphase ) 

t ime . max 

= 

[0 

constant  s 

tmax] 

limits ( iphase ) 

state . min 

(1 

) 

) 

=  [xlmin 

xlmin 

xlmin] 

limits ( iphase ) 

state . max 

(1 

) 

) 

=  [xlmax 

xlmax 

xlmax] 

limits ( iphase ) 

state . min 

(2 

9 

) 

=  [x2min 

x2min 

x2min] 

limits ( iphase ) 

state . max 

(2 

9 

) 

=  [x2max 

x2max 

x2max] 

limits ( iphase ) 

state . min 

(3 

9 

) 

=  [x3min 

x3min 

x3min] 

limits ( iphase ) 

state . max 

(3 

9 

) 

=  [x3max 

x3max 

x3max] 

limits ( iphase ) 

. control . min 

= 

-470/(11326/6076.11549) 

limits ( iphase ) 

. control . max 

= 

470/(113266076 . 11549)  ; 

limits ( iphase ) 

. parameter . min 

= 

par  am 

_min ; 

limits ( iphase ) 

. parameter . max 

= 

par  am 

_max ; 

limits ( iphase ) 

. path . min 

= 

path_ 

min ; 

limits ( iphase ) 

. path . max 

= 

path_ 

max  ; 

limits ( iphase ) 

. event .min 

= 

event 

_min ; 

limits ( iphase ) 

. event . max 

= 

event 

_max ; 

limits ( iphase ) 

. dur at i on . min 

=  []  ; 

limits ( iphase ) 

. dur at  ion . max 

=  []  ; 

guess ( iphase ) . 

time 

=  [0; 

0]  ; 

guess ( iphase ) . 

state ( : , 1 ) 

= 

[xlO; 

xlf]  ; 

guess ( iphase ) . 

state ( : ,2) 

= 

[x20  ; 

x2f  ]  ; 

guess ( iphase ) . 

state ( :  ,3) 

= 

[x30  ; 

x3f]  ; 

guess ( iphase ). control  =  [0;  0]; 

guess ( iphase ). parameter  =  [] ; 

clear  xlO  x20  x30  xlf  x2f  x3f  xlmin  xlmax  x2min  x2max  x3min  x3max 
clear  param.min  param_max  path_min  path_max  event.min  event_max 
clear  duration.min  duration.max  iphase 

setup. name  =  ’ My-Problem ’ ; 
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setup . funcs . cost  =  ’ MyProblemCost ’ ; 

setup . funcs . dae  =  1 MyProblemDae ’ ; 

setup . funcs . event  =  1 MyProblemEvent ’ ; 

setup. limits  =  limits; 

setup. guess  =  guess; 

setup . linkages  =  [] ; 

setup . derivatives  =  ’automatic’; 

setup  .  direction  =  ’decreasing’;  '/.increasing 

setup . autoscale  =  ’on’; 

output  =  gpops ( setup ) ; 
solution  =  output . solution ; 

xlout=solution. state ( : ,1) ; 
x2out=solution. state ( : ,2) ; 
x3out=solution. state ( : ,3) ; 
plout=solution. costateC : ,1) ; 
p2out  =  solution.costate(:  ,2)  ; 
p3out  =  solution. costateC:  ,3)  ; 
uout=solution . control; 
tout=solution. time ; 

Hamout=solution. Hamiltonian; 

costout=solution. Mayer_cost+solution. Lagrange.cost ; 


7. - 

7„  END:  script  MyProblemMain .  m 

7 - 


2.2  GPOPS  Cost  Function:  MyProblemCost. m 


Listing  B.2.  GPOPS  cost  function 

l - 

1  BEGIN:  function  MyProblemCost . m 

l - 

f unct ion  [Mayer, Lagrange , DMayer , DLag range] = MyProblemCost (sol) 
global  constants  ac  be  cc  PK  t  Jscl 


to 

= 

sol . initial . time ; 

xO 

= 

sol . initial . state ; 

tf 

= 

sol . terminal . time ; 

xf 

= 

sol . terminal . state 

t 

= 

sol . t ime ; 

X 

= 

sol . state  ; 

u 

= 

sol . control ; 

p 

= 

sol . parameter ; 

Mayer 

=  0 . 00001*tf  ; 

°/0  Sampled  values  (collocation  or  random)  are  passed  into  this 
funciton  as  ac ,  be,  cc 
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clx=constants . clxl+ac ;  cly=constants . clx2+ac ; 
c2x=constants . c2xl+bc ;  c2y=constants . c2x2+bc ; 
c3x=constants . c3xl+cc ;  c3y=constants . c3x2+cc ; 

7.  Threat  #1 

f 1 1  =  ( 1 / (2*pi*constants. sigtlxl*constants. sigtlx2*sqrt (1-constants.  .  .  . 
rho 1 ~2) ) ) ; 

fl2=l/ (2* (1-constants . rho 1 ~  2 ) ) ; 

1 1 1  =  ( x ( :  ,l)-clx)  . ~2 . /(const an ts.sigtlxl"2)  ; 

1 12  =  ( x ( :  ,2)-cly)  . ~  2 . /(const  an ts.sigtlx2~2)  ; 

tl3=(2*constants.rhol*(x(: ,l)-clx) .  *  (  x  (  : ,2)-cly)) ./(constants.... 
sigtlxl*constants. sigtlx2) ; 

PKl  =  f 11 . *exp(-f 12  .  *(tll  +  tl2-tl3))  ; 

7.  Threat  #2 

f 2 1  =  ( 1 / (2*pi*constants. sigt2xl*constants. sigt2x2*sqrt (1-constants.  .  .  . 
rho2  ~  2) ) )  ; 

f22=l/ (2* (1-const  an ts.rho2~2)) ; 

1 2 1  =  ( x ( :  ,l)-c2x)  . ~  2 . /(const  an ts.sigt2xl~2)  ; 
t22=(x(:  ,2)-c2y)  . ~  2 . /(const  an ts.sigt2x2~2)  ; 

t23=(2*constants.rho2*(x(: ,l)-c2x) .  *  (  x  (  : ,2)-c2y)) ./(constants.... 
sigt2xl*constants . sigt2x2) ; 

PK2  =  f 21 . *exp(-f 2  2  .  * (t21  +  t22-t23) ) ; 

7.  Threat  #3 

f31  =  (l/ (2*pi*constants. sigt3xl*constants. sigt3x2*sqrt (1-constants.  .  .  . 
rho3  ~  2) ) )  ; 

f32=l/ (2*(l-constants.rho3~2)) ; 

t3 1  =  ( x ( :  ,l)-c3x)  . ~  2 . /(const  an ts.sigt3xl~2)  ; 

t32=(x(:  ,2)-c3y)  . ~  2 . /(const  an ts.sigt3x2~2)  ; 

t33=(2*constants.rho3*(x(: ,l)-c3x) ,*(x(: ,2)-c3y)) ./(constants.... 
sigt3xl*constants . sigt3x2) ; 

PK3=f31 . *exp(-f32 . * (t31+t32-t33) ) ; 

Lagrange  =  (PK1+PK2+PK3) / Jscl ; 


7. - 

7,  END:  function  MyProblemCost  .  m 

7. - 


2.3  GPOPS  Differential  Algebraic  Equations  Function:  MyProblemDAE.m 


Listing  B.3.  GPOPS  DAE  function 

7. - 

7t  BEGIN:  function  MyProblemDae  .  m 

7. - 

function  dae  =  MyProblemDae ( sol ) ; 
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global  ac  be  constants 


t  =  sol . t ime  ; 
x  =  sol . state  ; 
u  =  sol . control ; 


xldot  =  constants . v . *cos (x  (:,  3)  )  ; 
x2dot  =  constants. v.*sin(x(:,  3)); 
x3dot  =  11 ; 

path  =  []  ; 

dae  =  [xldot  x2dot  x3dot  path] ; 


7. - 

7„  END:  function  MyPr  oblemDae  .  m 
1 - 


2.4  GPOPS  Event  Function:  MyProblemEvent.m 


Listing  B.4.  GPOPS  Event  Function 

v. - 

*/.  BEGIN:  function  MyProblemEvent.m 

/ - 

function  event  =  MyProblemEvent ( sol ) ; 
global  constants 

tO  =  sol  .  init ial . t ime ; 
xO  =  sol . init ial . state  ; 
tf  =  sol . terminal . time ; 
xf  =  sol . terminal . state ; 

event  =  [xO(l,:);  x0(2,:);  x0(3,:);  xf(l,:);  xf(2,:)] 


/ - 

*/.  END:  function  MyProblemEvent.m 

7 - 
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Appendix  C.  Monte-Carlo  Simulation  Code 


Selected  lines  of  MATLAB®code  are  given  in  this  appendix  to  show  how  the  MCS 
results  were  generated  for  comparison  with  the  hybrid  GPM-gPC  results  in 
Chapter  V.  A  similar  version  was  used  to  generate  the  MCS  results  presented  in 
Chapter  IV. 


Listing  C.l.  MCS  code 

7,  Initialize  variables  for  data  storage 
7,  States  x_l  ,  x_2  ,  and  x_3 
xlMC=  []  ;  x2MC=  []  ;  x3MC=[]; 

7»  Control  u 
uMC=  []  ; 

7.  Cost  J 
costMC  =  []  ; 

7.  Time 
tMC=  []  ; 

7.  Point  counter  for  plotting 
ptcount=  []  ; 

"/,  Initialize  variables  for  convergence  checks 

meanO ldX l=zer os ( 1 , Nnodes +2 ) ;  me an01dX2=zer os ( 1 , Nnodes +2 ) ;  ... 

mean01dX3=ones (1 ,Nnodes+2) ; 

convStop=0 ; 
chkPt  =500 ; 
numPts=  100000; 

7.  Generate  arrays  of  random  samples  for  a,  b,  c 
randsampG  =  randn  (numPts  ,  3)  ;  ‘/0  Gaussian 

randsampU=rand  (numPts  ,  3)  ;  '/,  Uniform 

7.  Initialize  counter 
ii  =  1; 

while  convStop==0  &&  ii<=numPts 

7.  For  Gaussian  distribut  ions  ...  scale  points  into  specified... 
distribution 

ac=mu+dev*  rands ampG(ii , 1)  ; 
bc=mu+dev* rands ampG ( ii ,2) ; 
cc=mu+dev* rands ampG ( ii ,3) ; 

7.  For  uniform  distribut  ions  ...  scale  points  into  specified  ... 
domain  [lb , ub] 
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ac=(lb)+(((ub)-(lb))*randsampU(ii ,1)) ; 
bc=(lb)+(((ub)-(lb))*randsampU(ii ,2)) ; 
cc=(lb)+(((ub)-(lb))*randsampU(ii ,3)) ; 

%  Call  GPOPS  main  script 
MyPr oblemMain ; 

•/.  Save  GPOPS  outputs 

7.  States  x_l,  x_2  ,  and  x_3 
xlMC= [xlMC ; xlout  ’]  ; 
x2MC= [x2MC ; x2out  ’]  ; 
x3MC= [x3MC ; x3out  ’]  ; 

'/,  Control  u 
uMC= [uMC ; uout ’ ] ; 

7,  Time 

tMC=  [tMC ; tout  ’  ]  ; 

7,  Cost  J 

costMC=[costMC; costout] ; 

7.  Point  counter 
ptcount=[ptcount; ii] ; 

7.  Check  convergence 

if  mod ( ii , 500) ==0 

convChkXl= ( abs (meanOldXl -mean (xlMC) ) ) ; 
convChkX2=(abs (mean01dX2-mean(x2MC) ) ) ; 
convChkX3=(abs (mean01dX3-mean(x3MC) ) ) ; 

if  max ( convChkXl)  <  =  0 . 0001  &&  max ( convChkX2)  <  =  0 . 000  1  . 

max ( c onvChkX3 ) <=0 . 000 1  &&  ii<numPts 
convStop=l ; 

end 

l  Save  data  for  next  convergence  check 
mean01dXl=mean (xlMC) ; 
mean01dX2=mean (x2MC) ; 
mean01dX3=mean (x3MC) ; 

end 
i  i  =  i  i  + 1  ; 

end 

7.  Save  data  to  .mat  files 
7.  States  x_l  ,  x_2  ,  and  x_3 
save  xlMC  xlMC 
save  x2MC  x2MC 
save  x3MC  x3MC 
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“/,  Control  u 
save  uMC  uMC 

1  Cost  J 

save  costMC  costMC 
%  Time 

save  tMC  tMC 


save  ptcount  ptcount 
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